2024-04-25T10:36:57.373 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:36:57.374 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096 -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/orig.config.yaml 2024-04-25T10:36:57.434 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:36:57.533 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673096' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJKZcuDuWUa+HBX9tJK0nlGzYOVV/g9qv57hNLGA3H3pgbcFakrmiXnQkNgrAuVHnHND7vi+kd3j6ls543STguE= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9mwjUhXnyYGuXrgGyayPZxk8caVtVbx02JxWK6GHP12aWYbov0MnZyJ3+QD0+BsxBnwwolJVxzouRy6fnBHRs= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+0DN1Ngt5nUosQFYmjtm6+1LSiabRiF6ZPwqMTXBnkqOJSfdkUKGbEzo6/NlBHBs4qOujbjuG0LB1LhhnXciM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:36:57.533 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T10:36:57.534 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T10:36:57.534 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:36:57.535 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:36:57.562 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T10:36:57.562 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:36:57.562 INFO:teuthology.packaging:ref: None 2024-04-25T10:36:57.562 INFO:teuthology.packaging:tag: None 2024-04-25T10:36:57.562 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:36:57.562 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:36:57.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:36:57.710 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T10:36:57.711 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:36:57.718 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:36:57.718 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:36:57.765 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:36:57.788 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:36:57.794 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:36:57.819 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:31:45.010810', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJKZcuDuWUa+HBX9tJK0nlGzYOVV/g9qv57hNLGA3H3pgbcFakrmiXnQkNgrAuVHnHND7vi+kd3j6ls543STguE='} 2024-04-25T10:36:57.842 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:31:45.013733', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9mwjUhXnyYGuXrgGyayPZxk8caVtVbx02JxWK6GHP12aWYbov0MnZyJ3+QD0+BsxBnwwolJVxzouRy6fnBHRs='} 2024-04-25T10:36:57.862 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:31:45.012592', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+0DN1Ngt5nUosQFYmjtm6+1LSiabRiF6ZPwqMTXBnkqOJSfdkUKGbEzo6/NlBHBs4qOujbjuG0LB1LhhnXciM='} 2024-04-25T10:36:57.862 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:36:57.868 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T10:36:57.868 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T10:36:57.868 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T10:36:57.868 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:36:57.988 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9619c670d0>, signals=[15]) 2024-04-25T10:36:57.988 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:36:57.994 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:36:57.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:36:57.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:36:58.071 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T10:36:58.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:36:58.145 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:36:58.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:36:58.225 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:36:58.232 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-04-25T10:36:58.251 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-04-25T10:36:58.251 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-04-25T10:36:58.308 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-04-25T10:36:58.308 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:36:58.310 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-04-25T10:36:58.332 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-25T10:36:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-25T10:36:58.350 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-25T10:36:58.407 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:36:58.408 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:36:58.408 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-25T10:36:58.432 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-25T10:36:58.449 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-25T10:36:58.449 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:36:58.507 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:36:58.508 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:36:58.508 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:36:58.508 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:36:58.508 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:36:58.508 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-25T10:36:58.531 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:36:58.542 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:36:58.547 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:36:58.547 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:36:58.550 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:36:58.553 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:36:58.569 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:36:58.577 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:36:58.578 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:36:58.608 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:36:58.610 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:36:58.718 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:36:58.795 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:36:58.809 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T10:36:58.809 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T10:36:58.809 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T10:36:58.809 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:36:58.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:36:58.827 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-25T10:36:58.884 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:36:58.885 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-437.el9.x86_64 2024-04-25T10:36:58.885 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-04-25T10:37:01.375 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-04-25T10:37:03.175 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-25T10:37:07.593 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-04-25T10:37:09.293 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-25T10:37:10.381 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-25T10:37:15.603 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T10:37:17.626 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:37:17.626 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:37:17.728 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T10:37:17.729 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-04-25T10:37:17.729 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T10:37:17.832 DEBUG:teuthology.orchestra.run.smithi019:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:37:18.356 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:37:15 AM UTC. 2024-04-25T10:37:18.490 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T10:37:18.499 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:37:18.500 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-25T10:37:18.500 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-25T10:37:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:Operation aborted. 2024-04-25T10:37:18.561 INFO:teuthology.orchestra.run.smithi019.stdout:Is this ok [y/N]: 2024-04-25T10:37:18.562 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum reinstall -y kernel || true 2024-04-25T10:37:19.081 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 10:37:15 AM UTC. 2024-04-25T10:37:19.201 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T10:37:19.201 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-25T10:37:19.202 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T10:37:19.355 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.9 MB/s | 255 kB 00:00 2024-04-25T10:37:19.355 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:37:19.355 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.6 MB/s | 255 kB 00:00 2024-04-25T10:37:19.384 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T10:37:19.574 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T10:37:19.575 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T10:37:19.580 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T10:37:19.580 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T10:37:19.719 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T10:37:19.823 INFO:teuthology.orchestra.run.smithi019.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:37:19.823 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:37:20.514 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:37:20.515 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:37:21.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:37:21.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:37:21.262 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalled: 2024-04-25T10:37:21.262 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:37:21.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:37:21.262 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T10:37:21.447 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:37:21.496 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:37:21.496 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:37:21.496 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:37:21.497 DEBUG:teuthology.parallel:result is None 2024-04-25T10:37:21.497 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:37:21.507 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:37:21.507 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:37:21.509 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:37:21.512 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:37:21.531 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:37:21.540 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:37:21.548 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:37:21.549 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:37:21.565 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:37:21.569 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:37:21.666 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:37:21.674 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:37:21.674 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:37:21.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:37:21.691 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:37:21.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:37:21.707 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:37:21.709 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:37:21.733 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:37:21.750 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:37:21.763 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:37:21.768 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:37:21.792 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:37:21.798 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:37:21.832 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:37:21.834 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:37:21.842 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:37:21.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:37:21.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:37:21.846 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:37:21.872 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:37:21.918 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:37:21.919 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:37:21.921 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:37:21.923 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:37:21.938 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:37:22.016 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:37:22.069 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:37:22.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:37:22.190 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:37:22.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:37:22.255 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:37:22.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:37:22.353 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:37:22.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:37:22.477 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:37:22.477 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:37:22.540 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:37:22.585 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:37:22.641 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:37:22.723 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:37:22.767 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:37:22.767 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:37:22.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-04-25T10:37:22.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-25T10:37:22.837 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-25T10:37:22.868 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:37:22.871 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:37:22.918 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:37:23.351 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:37:23.359 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:37:23.359 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:37:23.374 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:37:23.383 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T10:37:23.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-04-25T10:37:23.432 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-04-25T10:37:23.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-25T10:37:23.483 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-25T10:37:23.485 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-25T10:37:23.540 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-25T10:37:23.542 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:37:23.542 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-04-25T10:37:23.563 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-04-25T10:37:23.563 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-25T10:37:23.585 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-25T10:37:23.585 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-25T10:37:23.606 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-25T10:37:23.607 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-04-25T10:37:23.607 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:37:23.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:37:23.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:37:23.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:37:23.657 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:37:23.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:37:23.683 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:37:23.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:37:23.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:37:23.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:37:23.758 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:37:23.768 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:37:23.783 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:37:23.784 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi112.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:41:28.315 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-25T10:41:28.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T10:41:28.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:41:28.400 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-25T10:41:28.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T10:41:28.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T10:41:28.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:41:28.570 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T10:41:28.646 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T10:41:28.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T10:41:28.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:41:28.725 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T10:41:28.814 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T10:41:28.814 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:41:28.828 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:41:28.828 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:41:28.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:41:28.831 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:41:28.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:41:28.834 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:41:28.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:41:28.861 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:41:28.862 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:41:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:41:28.884 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:41:28.898 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:41:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-25T10:41:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-04-25T10:41:28.911 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-25T10:41:28.912 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-04-25T10:41:28.921 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-25T10:41:28.924 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:41:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:41:28.931 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-25T10:41:28.938 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:41:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:41:28.944 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:41:28.957 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:41:28.988 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:41:28.991 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:41:28.991 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T10:41:28.991 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:28.991 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:28.991 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:28.991 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:28.999 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:41:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:41:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T10:41:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.011 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:41:29.016 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:41:29.016 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T10:41:29.016 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.016 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.017 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.017 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:41:29.017 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:41:29.030 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:41:29.031 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T10:41:29.031 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T10:41:29.031 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:41:29.038 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:41:29.038 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:41:29.038 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:41:29.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:41:29.041 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:41:29.041 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:41:29.042 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:41:29.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:41:29.197 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:41:29.198 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:41:29.202 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:41:29.202 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:41:29.212 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:41:29.212 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:41:29.306 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:41:29.306 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:41:29.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:41:29.333 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:41:29.333 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:41:29.371 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:41:29.371 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:41:29.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:41:29.383 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:41:29.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:41:29.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:41:29.397 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:41:29.397 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:41:29.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:41:29.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:41:29.409 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:41:29.463 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:41:29.474 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:41:29.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:41:29.537 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:41:29.538 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-04-25T10:41:29.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-04-25T10:41:29.550 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:41:29.600 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-25T10:41:29.601 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-25T10:41:29.613 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-25T10:41:29.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-25T10:41:29.771 INFO:teuthology.orchestra.run.smithi019.stdout:48 files removed 2024-04-25T10:41:29.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:41:29.827 INFO:teuthology.orchestra.run.smithi169.stdout:48 files removed 2024-04-25T10:41:29.836 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-25T10:41:29.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:41:29.862 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:41:30.430 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 234 kB/s | 84 kB 00:00 2024-04-25T10:41:30.507 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 217 kB/s | 84 kB 00:00 2024-04-25T10:41:30.594 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 175 kB/s | 84 kB 00:00 2024-04-25T10:41:30.865 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T10:41:30.872 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-25T10:41:30.953 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T10:41:31.264 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-25T10:41:31.378 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-25T10:41:31.391 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-25T10:41:31.806 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-25T10:41:32.191 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T10:41:32.378 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-25T10:41:34.405 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T10:41:35.551 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-04-25T10:41:35.640 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 19 MB 00:02 2024-04-25T10:41:39.772 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-04-25T10:41:39.858 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T10:41:40.269 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-25T10:41:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T10:41:41.534 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T10:41:41.686 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-25T10:41:42.710 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T10:41:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-25T10:41:44.149 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-04-25T10:41:47.830 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-25T10:41:47.993 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T10:41:49.174 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-25T10:41:49.947 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:41:49.948 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:41:49.960 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:41:49.960 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:41:49.961 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:41:49.962 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:41:49.963 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:41:49.966 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:41:49.966 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:41:50.026 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:41:50.061 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:41:50.062 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:41:50.063 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:41:50.064 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:41:50.065 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:41:50.066 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:41:50.067 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:41:50.068 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:41:50.069 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:41:50.070 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:41:50.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:41:50.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T10:41:50.073 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T10:41:50.074 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-25T10:41:50.074 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 4 Packages 2024-04-25T10:41:50.074 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:41:50.074 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 214 M 2024-04-25T10:41:50.075 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T10:41:50.212 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:41:50.213 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:41:50.224 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:41:50.225 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:41:50.226 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:41:50.226 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:41:50.228 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:41:50.230 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:41:50.231 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:41:50.297 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T10:41:50.306 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T10:41:50.306 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-04-25T10:41:50.306 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T10:41:50.306 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-04-25T10:41:50.306 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:41:50.307 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:41:50.308 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:41:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:41:50.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:41:50.311 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:41:50.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:41:50.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:41:50.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:41:50.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:41:50.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:41:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T10:41:50.318 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-04-25T10:41:50.319 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 4 Packages 2024-04-25T10:41:50.319 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:41:50.319 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 214 M 2024-04-25T10:41:50.319 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T10:41:51.081 INFO:teuthology.orchestra.run.smithi112.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-25T10:41:51.228 INFO:teuthology.orchestra.run.smithi019.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T10:41:51.290 INFO:teuthology.orchestra.run.smithi112.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 4.0 MB/s | 863 kB 00:00 2024-04-25T10:41:51.308 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:41:51.308 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:41:51.320 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:41:51.321 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:41:51.321 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:41:51.323 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:41:51.325 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:41:51.326 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:41:51.348 INFO:teuthology.orchestra.run.smithi112.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 2.5 MB/s | 147 kB 00:00 2024-04-25T10:41:51.386 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T10:41:51.401 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T10:41:51.401 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-25T10:41:51.401 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:41:51.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:41:51.403 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:41:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:41:51.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:41:51.406 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:41:51.407 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:41:51.408 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:41:51.409 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:41:51.410 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:41:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:41:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 4 Packages 2024-04-25T10:41:51.413 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:41:51.414 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 214 M 2024-04-25T10:41:51.414 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-25T10:41:51.590 INFO:teuthology.orchestra.run.smithi112.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 7.9 MB/s | 5.4 MB 00:00 2024-04-25T10:41:51.682 INFO:teuthology.orchestra.run.smithi112.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 6.8 MB/s | 2.3 MB 00:00 2024-04-25T10:41:51.695 INFO:teuthology.orchestra.run.smithi019.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 863 kB 00:00 2024-04-25T10:41:51.816 INFO:teuthology.orchestra.run.smithi112.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-04-25T10:41:51.853 INFO:teuthology.orchestra.run.smithi019.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 929 kB/s | 147 kB 00:00 2024-04-25T10:41:52.333 INFO:teuthology.orchestra.run.smithi112.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 7.5 MB/s | 4.9 MB 00:00 2024-04-25T10:41:52.667 INFO:teuthology.orchestra.run.smithi169.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-25T10:41:53.492 INFO:teuthology.orchestra.run.smithi169.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 863 kB 00:00 2024-04-25T10:41:53.612 INFO:teuthology.orchestra.run.smithi019.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 1.3 MB/s | 2.3 MB 00:01 2024-04-25T10:41:53.675 INFO:teuthology.orchestra.run.smithi169.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 803 kB/s | 147 kB 00:00 2024-04-25T10:41:54.108 INFO:teuthology.orchestra.run.smithi112.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 5.7 MB/s | 10 MB 00:01 2024-04-25T10:41:54.183 INFO:teuthology.orchestra.run.smithi112.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 336 kB/s | 25 kB 00:00 2024-04-25T10:41:54.329 INFO:teuthology.orchestra.run.smithi019.stdout:(5/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-25T10:41:54.483 INFO:teuthology.orchestra.run.smithi112.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 6.7 MB/s | 18 MB 00:02 2024-04-25T10:41:54.550 INFO:teuthology.orchestra.run.smithi112.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 3.3 MB/s | 224 kB 00:00 2024-04-25T10:41:54.626 INFO:teuthology.orchestra.run.smithi112.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 360 kB/s | 27 kB 00:00 2024-04-25T10:41:54.709 INFO:teuthology.orchestra.run.smithi112.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T10:41:54.745 INFO:teuthology.orchestra.run.smithi019.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-04-25T10:41:54.867 INFO:teuthology.orchestra.run.smithi112.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.7 MB/s | 756 kB 00:00 2024-04-25T10:41:54.951 INFO:teuthology.orchestra.run.smithi112.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.8 MB/s | 158 kB 00:00 2024-04-25T10:41:55.026 INFO:teuthology.orchestra.run.smithi112.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T10:41:55.176 INFO:teuthology.orchestra.run.smithi112.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.2 MB/s | 497 kB 00:00 2024-04-25T10:41:55.242 INFO:teuthology.orchestra.run.smithi169.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-25T10:41:55.467 INFO:teuthology.orchestra.run.smithi169.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 1.3 MB/s | 2.3 MB 00:01 2024-04-25T10:41:55.521 INFO:teuthology.orchestra.run.smithi019.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 4.1 MB/s | 4.9 MB 00:01 2024-04-25T10:41:55.705 INFO:teuthology.orchestra.run.smithi019.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 4.2 MB/s | 20 MB 00:04 2024-04-25T10:41:55.796 INFO:teuthology.orchestra.run.smithi019.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 276 kB/s | 25 kB 00:00 2024-04-25T10:41:56.159 INFO:teuthology.orchestra.run.smithi169.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-04-25T10:41:56.568 INFO:teuthology.orchestra.run.smithi112.stdout:(18/152): ceph-common-19.0.0-3291.g994e2f20.el9 3.5 MB/s | 20 MB 00:05 2024-04-25T10:41:56.635 INFO:teuthology.orchestra.run.smithi112.stdout:(19/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 3.5 MB/s | 5.2 MB 00:01 2024-04-25T10:41:56.661 INFO:teuthology.orchestra.run.smithi112.stdout:(20/152): python3-ceph-argparse-19.0.0-3291.g99 491 kB/s | 45 kB 00:00 2024-04-25T10:41:56.703 INFO:teuthology.orchestra.run.smithi112.stdout:(21/152): python3-ceph-common-19.0.0-3291.g994e 2.0 MB/s | 135 kB 00:00 2024-04-25T10:41:56.753 INFO:teuthology.orchestra.run.smithi112.stdout:(22/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.7 MB/s | 161 kB 00:00 2024-04-25T10:41:56.828 INFO:teuthology.orchestra.run.smithi112.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 2.5 MB/s | 321 kB 00:00 2024-04-25T10:41:56.903 INFO:teuthology.orchestra.run.smithi112.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 100 kB 00:00 2024-04-25T10:41:56.929 INFO:teuthology.orchestra.run.smithi112.stdout:(25/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 298 kB 00:00 2024-04-25T10:41:56.971 INFO:teuthology.orchestra.run.smithi112.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 86 kB 00:00 2024-04-25T10:41:57.046 INFO:teuthology.orchestra.run.smithi112.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.2 MB/s | 169 kB 00:00 2024-04-25T10:41:57.122 INFO:teuthology.orchestra.run.smithi112.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 456 kB/s | 34 kB 00:00 2024-04-25T10:41:57.197 INFO:teuthology.orchestra.run.smithi112.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.9 MB/s | 142 kB 00:00 2024-04-25T10:41:57.405 INFO:teuthology.orchestra.run.smithi019.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 5.4 MB/s | 10 MB 00:01 2024-04-25T10:41:57.505 INFO:teuthology.orchestra.run.smithi019.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.2 MB/s | 224 kB 00:00 2024-04-25T10:41:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 324 kB/s | 27 kB 00:00 2024-04-25T10:41:57.680 INFO:teuthology.orchestra.run.smithi019.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 346 kB/s | 32 kB 00:00 2024-04-25T10:41:57.839 INFO:teuthology.orchestra.run.smithi112.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 5.5 MB/s | 3.5 MB 00:00 2024-04-25T10:41:57.889 INFO:teuthology.orchestra.run.smithi019.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.5 MB/s | 756 kB 00:00 2024-04-25T10:41:57.897 INFO:teuthology.orchestra.run.smithi112.stdout:(31/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.1 MB/s | 3.0 MB 00:00 2024-04-25T10:41:57.981 INFO:teuthology.orchestra.run.smithi019.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T10:41:58.014 INFO:teuthology.orchestra.run.smithi112.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.1 MB/s | 249 kB 00:00 2024-04-25T10:41:58.081 INFO:teuthology.orchestra.run.smithi019.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.2 MB/s | 127 kB 00:00 2024-04-25T10:41:58.123 INFO:teuthology.orchestra.run.smithi112.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 455 kB/s | 49 kB 00:00 2024-04-25T10:41:58.256 INFO:teuthology.orchestra.run.smithi112.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 123 kB/s | 16 kB 00:00 2024-04-25T10:41:58.273 INFO:teuthology.orchestra.run.smithi019.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.5 MB/s | 497 kB 00:00 2024-04-25T10:41:58.382 INFO:teuthology.orchestra.run.smithi112.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.1 MB/s | 273 kB 00:00 2024-04-25T10:41:58.684 INFO:teuthology.orchestra.run.smithi169.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 1.5 MB/s | 4.9 MB 00:03 2024-04-25T10:41:58.716 INFO:teuthology.orchestra.run.smithi112.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.2 MB/s | 762 kB 00:00 2024-04-25T10:41:58.816 INFO:teuthology.orchestra.run.smithi112.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 182 kB/s | 18 kB 00:00 2024-04-25T10:41:58.883 INFO:teuthology.orchestra.run.smithi112.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-04-25T10:41:58.992 INFO:teuthology.orchestra.run.smithi112.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-25T10:41:59.084 INFO:teuthology.orchestra.run.smithi112.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 5.9 MB/s | 7.4 MB 00:01 2024-04-25T10:41:59.142 INFO:teuthology.orchestra.run.smithi112.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-25T10:41:59.176 INFO:teuthology.orchestra.run.smithi112.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T10:41:59.243 INFO:teuthology.orchestra.run.smithi112.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T10:41:59.281 INFO:teuthology.orchestra.run.smithi019.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 5.1 MB/s | 5.2 MB 00:01 2024-04-25T10:41:59.301 INFO:teuthology.orchestra.run.smithi112.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:41:59.344 INFO:teuthology.orchestra.run.smithi112.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-25T10:41:59.365 INFO:teuthology.orchestra.run.smithi019.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 542 kB/s | 45 kB 00:00 2024-04-25T10:41:59.402 INFO:teuthology.orchestra.run.smithi112.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-25T10:41:59.444 INFO:teuthology.orchestra.run.smithi112.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-25T10:41:59.466 INFO:teuthology.orchestra.run.smithi019.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.3 MB/s | 135 kB 00:00 2024-04-25T10:41:59.528 INFO:teuthology.orchestra.run.smithi112.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-25T10:41:59.558 INFO:teuthology.orchestra.run.smithi019.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.7 MB/s | 161 kB 00:00 2024-04-25T10:41:59.595 INFO:teuthology.orchestra.run.smithi112.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 943 kB 00:00 2024-04-25T10:41:59.620 INFO:teuthology.orchestra.run.smithi112.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-25T10:41:59.641 INFO:teuthology.orchestra.run.smithi019.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 3.8 MB/s | 321 kB 00:00 2024-04-25T10:41:59.670 INFO:teuthology.orchestra.run.smithi112.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-25T10:41:59.729 INFO:teuthology.orchestra.run.smithi112.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-04-25T10:41:59.733 INFO:teuthology.orchestra.run.smithi019.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T10:41:59.754 INFO:teuthology.orchestra.run.smithi112.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-25T10:41:59.805 INFO:teuthology.orchestra.run.smithi112.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-25T10:41:59.817 INFO:teuthology.orchestra.run.smithi019.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.2 MB/s | 100 kB 00:00 2024-04-25T10:41:59.846 INFO:teuthology.orchestra.run.smithi112.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-25T10:41:59.901 INFO:teuthology.orchestra.run.smithi019.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 86 kB 00:00 2024-04-25T10:41:59.922 INFO:teuthology.orchestra.run.smithi112.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T10:41:59.981 INFO:teuthology.orchestra.run.smithi112.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-04-25T10:42:00.064 INFO:teuthology.orchestra.run.smithi112.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.9 MB/s | 3.0 MB 00:00 2024-04-25T10:42:00.140 INFO:teuthology.orchestra.run.smithi112.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-25T10:42:00.181 INFO:teuthology.orchestra.run.smithi112.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-25T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 379 kB/s | 25 kB 00:00 2024-04-25T10:42:00.286 INFO:teuthology.orchestra.run.smithi112.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.6 MB/s | 292 kB 00:00 2024-04-25T10:42:00.349 INFO:teuthology.orchestra.run.smithi112.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 283 kB/s | 47 kB 00:00 2024-04-25T10:42:00.374 INFO:teuthology.orchestra.run.smithi112.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 188 kB 00:00 2024-04-25T10:42:00.416 INFO:teuthology.orchestra.run.smithi112.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-04-25T10:42:00.434 INFO:teuthology.orchestra.run.smithi019.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 5.7 MB/s | 3.0 MB 00:00 2024-04-25T10:42:00.501 INFO:teuthology.orchestra.run.smithi019.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.5 MB/s | 169 kB 00:00 2024-04-25T10:42:00.576 INFO:teuthology.orchestra.run.smithi019.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 456 kB/s | 34 kB 00:00 2024-04-25T10:42:00.652 INFO:teuthology.orchestra.run.smithi019.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.9 MB/s | 142 kB 00:00 2024-04-25T10:42:00.675 INFO:teuthology.orchestra.run.smithi112.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-04-25T10:42:00.742 INFO:teuthology.orchestra.run.smithi112.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T10:42:00.867 INFO:teuthology.orchestra.run.smithi112.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-04-25T10:42:00.909 INFO:teuthology.orchestra.run.smithi112.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-04-25T10:42:00.968 INFO:teuthology.orchestra.run.smithi112.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-25T10:42:01.085 INFO:teuthology.orchestra.run.smithi112.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-25T10:42:01.152 INFO:teuthology.orchestra.run.smithi112.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T10:42:01.161 INFO:teuthology.orchestra.run.smithi019.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 7.0 MB/s | 3.5 MB 00:00 2024-04-25T10:42:01.227 INFO:teuthology.orchestra.run.smithi112.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-25T10:42:01.310 INFO:teuthology.orchestra.run.smithi112.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T10:42:01.394 INFO:teuthology.orchestra.run.smithi112.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-25T10:42:01.436 INFO:teuthology.orchestra.run.smithi112.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-04-25T10:42:01.520 INFO:teuthology.orchestra.run.smithi112.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-25T10:42:01.553 INFO:teuthology.orchestra.run.smithi112.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 163 kB/s | 19 kB 00:00 2024-04-25T10:42:01.612 INFO:teuthology.orchestra.run.smithi112.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-25T10:42:01.629 INFO:teuthology.orchestra.run.smithi112.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 459 kB/s | 35 kB 00:00 2024-04-25T10:42:01.705 INFO:teuthology.orchestra.run.smithi112.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-25T10:42:01.780 INFO:teuthology.orchestra.run.smithi112.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-25T10:42:01.847 INFO:teuthology.orchestra.run.smithi112.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T10:42:01.914 INFO:teuthology.orchestra.run.smithi112.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-25T10:42:01.964 INFO:teuthology.orchestra.run.smithi112.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-04-25T10:42:02.014 INFO:teuthology.orchestra.run.smithi112.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T10:42:02.048 INFO:teuthology.orchestra.run.smithi112.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-25T10:42:02.098 INFO:teuthology.orchestra.run.smithi112.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-25T10:42:02.123 INFO:teuthology.orchestra.run.smithi112.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-04-25T10:42:02.161 INFO:teuthology.orchestra.run.smithi019.stdout:(31/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 2.4 MB/s | 18 MB 00:07 2024-04-25T10:42:02.253 INFO:teuthology.orchestra.run.smithi019.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-04-25T10:42:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.1 MB/s | 249 kB 00:00 2024-04-25T10:42:02.320 INFO:teuthology.orchestra.run.smithi019.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 736 kB/s | 49 kB 00:00 2024-04-25T10:42:02.345 INFO:teuthology.orchestra.run.smithi019.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 245 kB/s | 16 kB 00:00 2024-04-25T10:42:02.396 INFO:teuthology.orchestra.run.smithi019.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 3.6 MB/s | 273 kB 00:00 2024-04-25T10:42:02.462 INFO:teuthology.orchestra.run.smithi019.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 274 kB/s | 18 kB 00:00 2024-04-25T10:42:02.524 INFO:teuthology.orchestra.run.smithi112.stdout:(90/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 5.2 MB/s | 43 MB 00:08 2024-04-25T10:42:02.549 INFO:teuthology.orchestra.run.smithi112.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 119 kB/s | 54 kB 00:00 2024-04-25T10:42:02.554 INFO:teuthology.orchestra.run.smithi019.stdout:(38/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 3.6 MB/s | 762 kB 00:00 2024-04-25T10:42:02.599 INFO:teuthology.orchestra.run.smithi112.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 556 kB/s | 42 kB 00:00 2024-04-25T10:42:02.625 INFO:teuthology.orchestra.run.smithi112.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-04-25T10:42:02.671 INFO:teuthology.orchestra.run.smithi019.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 385 kB/s | 80 kB 00:00 2024-04-25T10:42:02.675 INFO:teuthology.orchestra.run.smithi112.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-04-25T10:42:02.700 INFO:teuthology.orchestra.run.smithi112.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-25T10:42:02.750 INFO:teuthology.orchestra.run.smithi112.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-04-25T10:42:02.763 INFO:teuthology.orchestra.run.smithi019.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-25T10:42:02.767 INFO:teuthology.orchestra.run.smithi169.stdout:(8/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x86 2.7 MB/s | 18 MB 00:06 2024-04-25T10:42:02.793 INFO:teuthology.orchestra.run.smithi112.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-25T10:42:02.818 INFO:teuthology.orchestra.run.smithi112.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-04-25T10:42:02.855 INFO:teuthology.orchestra.run.smithi019.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T10:42:02.876 INFO:teuthology.orchestra.run.smithi112.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T10:42:02.889 INFO:teuthology.orchestra.run.smithi019.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-04-25T10:42:02.902 INFO:teuthology.orchestra.run.smithi112.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-04-25T10:42:02.960 INFO:teuthology.orchestra.run.smithi112.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-04-25T10:42:02.976 INFO:teuthology.orchestra.run.smithi169.stdout:(9/152): ceph-common-19.0.0-3291.g994e2f20.el9. 1.8 MB/s | 20 MB 00:10 2024-04-25T10:42:02.985 INFO:teuthology.orchestra.run.smithi112.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-04-25T10:42:03.001 INFO:teuthology.orchestra.run.smithi169.stdout:(10/152): ceph-selinux-19.0.0-3291.g994e2f20.el 108 kB/s | 25 kB 00:00 2024-04-25T10:42:03.109 INFO:teuthology.orchestra.run.smithi169.stdout:(11/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 2.3 MB/s | 10 MB 00:04 2024-04-25T10:42:03.135 INFO:teuthology.orchestra.run.smithi169.stdout:(12/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.6 MB/s | 224 kB 00:00 2024-04-25T10:42:03.168 INFO:teuthology.orchestra.run.smithi169.stdout:(13/152): cephfs-shell-19.0.0-3291.g994e2f20.el 463 kB/s | 27 kB 00:00 2024-04-25T10:42:03.225 INFO:teuthology.orchestra.run.smithi112.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-25T10:42:03.225 INFO:teuthology.orchestra.run.smithi169.stdout:(14/152): libcephfs-devel-19.0.0-3291.g994e2f20 543 kB/s | 32 kB 00:00 2024-04-25T10:42:03.289 INFO:teuthology.orchestra.run.smithi019.stdout:(43/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 5.8 MB/s | 43 MB 00:07 2024-04-25T10:42:03.294 INFO:teuthology.orchestra.run.smithi112.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-25T10:42:03.314 INFO:teuthology.orchestra.run.smithi019.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 250 kB/s | 106 kB 00:00 2024-04-25T10:42:03.323 INFO:teuthology.orchestra.run.smithi169.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.2 MB/s | 158 kB 00:00 2024-04-25T10:42:03.373 INFO:teuthology.orchestra.run.smithi019.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T10:42:03.394 INFO:teuthology.orchestra.run.smithi169.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T10:42:03.415 INFO:teuthology.orchestra.run.smithi019.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-04-25T10:42:03.428 INFO:teuthology.orchestra.run.smithi169.stdout:(17/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 2.8 MB/s | 756 kB 00:00 2024-04-25T10:42:03.440 INFO:teuthology.orchestra.run.smithi019.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-25T10:42:03.470 INFO:teuthology.orchestra.run.smithi112.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 112 kB/s | 19 kB 00:00 2024-04-25T10:42:03.499 INFO:teuthology.orchestra.run.smithi019.stdout:(48/152): python3-pycparser-2.20-6.el9.noarch.r 646 kB/s | 135 kB 00:00 2024-04-25T10:42:03.536 INFO:teuthology.orchestra.run.smithi169.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 3.4 MB/s | 497 kB 00:00 2024-04-25T10:42:03.549 INFO:teuthology.orchestra.run.smithi019.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-04-25T10:42:03.574 INFO:teuthology.orchestra.run.smithi019.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-04-25T10:42:03.600 INFO:teuthology.orchestra.run.smithi019.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-25T10:42:03.603 INFO:teuthology.orchestra.run.smithi169.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 678 kB/s | 45 kB 00:00 2024-04-25T10:42:03.616 INFO:teuthology.orchestra.run.smithi019.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 319 kB/s | 21 kB 00:00 2024-04-25T10:42:03.650 INFO:teuthology.orchestra.run.smithi019.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-25T10:42:03.662 INFO:teuthology.orchestra.run.smithi169.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.3 MB/s | 135 kB 00:00 2024-04-25T10:42:03.675 INFO:teuthology.orchestra.run.smithi019.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-04-25T10:42:03.709 INFO:teuthology.orchestra.run.smithi019.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-25T10:42:03.734 INFO:teuthology.orchestra.run.smithi019.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-25T10:42:03.745 INFO:teuthology.orchestra.run.smithi169.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.9 MB/s | 161 kB 00:00 2024-04-25T10:42:03.767 INFO:teuthology.orchestra.run.smithi019.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T10:42:03.793 INFO:teuthology.orchestra.run.smithi019.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-25T10:42:03.843 INFO:teuthology.orchestra.run.smithi019.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-25T10:42:03.854 INFO:teuthology.orchestra.run.smithi169.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 2.9 MB/s | 321 kB 00:00 2024-04-25T10:42:03.902 INFO:teuthology.orchestra.run.smithi019.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-04-25T10:42:03.903 INFO:teuthology.orchestra.run.smithi112.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 597 kB/s | 548 kB 00:00 2024-04-25T10:42:03.943 INFO:teuthology.orchestra.run.smithi019.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-25T10:42:03.954 INFO:teuthology.orchestra.run.smithi169.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.9 MB/s | 298 kB 00:00 2024-04-25T10:42:03.962 INFO:teuthology.orchestra.run.smithi112.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 394 kB/s | 309 kB 00:00 2024-04-25T10:42:03.969 INFO:teuthology.orchestra.run.smithi019.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-04-25T10:42:03.994 INFO:teuthology.orchestra.run.smithi019.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-04-25T10:42:04.023 INFO:teuthology.orchestra.run.smithi169.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.4 MB/s | 100 kB 00:00 2024-04-25T10:42:04.027 INFO:teuthology.orchestra.run.smithi019.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T10:42:04.054 INFO:teuthology.orchestra.run.smithi019.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-25T10:42:04.078 INFO:teuthology.orchestra.run.smithi019.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 419 kB/s | 35 kB 00:00 2024-04-25T10:42:04.082 INFO:teuthology.orchestra.run.smithi169.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.4 MB/s | 86 kB 00:00 2024-04-25T10:42:04.187 INFO:teuthology.orchestra.run.smithi019.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T10:42:04.296 INFO:teuthology.orchestra.run.smithi019.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T10:42:04.338 INFO:teuthology.orchestra.run.smithi019.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.9 MB/s | 911 kB 00:00 2024-04-25T10:42:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T10:42:04.438 INFO:teuthology.orchestra.run.smithi019.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-25T10:42:04.497 INFO:teuthology.orchestra.run.smithi019.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T10:42:04.630 INFO:teuthology.orchestra.run.smithi019.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-25T10:42:04.633 INFO:teuthology.orchestra.run.smithi169.stdout:(26/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 4.3 MB/s | 5.2 MB 00:01 2024-04-25T10:42:04.689 INFO:teuthology.orchestra.run.smithi019.stdout:(74/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-04-25T10:42:04.700 INFO:teuthology.orchestra.run.smithi169.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.5 MB/s | 169 kB 00:00 2024-04-25T10:42:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T10:42:04.747 INFO:teuthology.orchestra.run.smithi019.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-25T10:42:04.750 INFO:teuthology.orchestra.run.smithi169.stdout:(28/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 4.6 MB/s | 3.0 MB 00:00 2024-04-25T10:42:04.764 INFO:teuthology.orchestra.run.smithi019.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-25T10:42:04.775 INFO:teuthology.orchestra.run.smithi169.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 455 kB/s | 34 kB 00:00 2024-04-25T10:42:04.809 INFO:teuthology.orchestra.run.smithi169.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.4 MB/s | 142 kB 00:00 2024-04-25T10:42:04.857 INFO:teuthology.orchestra.run.smithi019.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-25T10:42:05.015 INFO:teuthology.orchestra.run.smithi019.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 19 kB 00:00 2024-04-25T10:42:05.040 INFO:teuthology.orchestra.run.smithi019.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 189 kB/s | 35 kB 00:00 2024-04-25T10:42:05.065 INFO:teuthology.orchestra.run.smithi019.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 573 kB/s | 172 kB 00:00 2024-04-25T10:42:05.099 INFO:teuthology.orchestra.run.smithi019.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T10:42:05.133 INFO:teuthology.orchestra.run.smithi019.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-25T10:42:05.158 INFO:teuthology.orchestra.run.smithi019.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T10:42:05.192 INFO:teuthology.orchestra.run.smithi019.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-04-25T10:42:05.217 INFO:teuthology.orchestra.run.smithi019.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:42:05.250 INFO:teuthology.orchestra.run.smithi019.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T10:42:05.276 INFO:teuthology.orchestra.run.smithi019.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T10:42:05.309 INFO:teuthology.orchestra.run.smithi019.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 875 kB/s | 51 kB 00:00 2024-04-25T10:42:05.335 INFO:teuthology.orchestra.run.smithi019.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-04-25T10:42:05.393 INFO:teuthology.orchestra.run.smithi019.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-04-25T10:42:05.501 INFO:teuthology.orchestra.run.smithi169.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 4.9 MB/s | 3.5 MB 00:00 2024-04-25T10:42:05.527 INFO:teuthology.orchestra.run.smithi019.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 332 kB/s | 44 kB 00:00 2024-04-25T10:42:05.610 INFO:teuthology.orchestra.run.smithi169.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.2 MB/s | 249 kB 00:00 2024-04-25T10:42:05.636 INFO:teuthology.orchestra.run.smithi019.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-04-25T10:42:05.661 INFO:teuthology.orchestra.run.smithi019.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-04-25T10:42:05.694 INFO:teuthology.orchestra.run.smithi019.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-25T10:42:05.720 INFO:teuthology.orchestra.run.smithi019.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-25T10:42:05.735 INFO:teuthology.orchestra.run.smithi169.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 395 kB/s | 49 kB 00:00 2024-04-25T10:42:05.762 INFO:teuthology.orchestra.run.smithi019.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-25T10:42:05.787 INFO:teuthology.orchestra.run.smithi019.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-04-25T10:42:05.802 INFO:teuthology.orchestra.run.smithi169.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 247 kB/s | 16 kB 00:00 2024-04-25T10:42:05.820 INFO:teuthology.orchestra.run.smithi019.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:42:05.854 INFO:teuthology.orchestra.run.smithi019.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 950 kB/s | 64 kB 00:00 2024-04-25T10:42:05.888 INFO:teuthology.orchestra.run.smithi019.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 668 kB/s | 45 kB 00:00 2024-04-25T10:42:05.969 INFO:teuthology.orchestra.run.smithi169.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.6 MB/s | 273 kB 00:00 2024-04-25T10:42:06.055 INFO:teuthology.orchestra.run.smithi019.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-25T10:42:06.080 INFO:teuthology.orchestra.run.smithi019.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-04-25T10:42:06.169 INFO:teuthology.orchestra.run.smithi169.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-04-25T10:42:06.172 INFO:teuthology.orchestra.run.smithi019.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-04-25T10:42:06.197 INFO:teuthology.orchestra.run.smithi019.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.8 MB/s | 548 kB 00:00 2024-04-25T10:42:06.203 INFO:teuthology.orchestra.run.smithi169.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 3.2 MB/s | 762 kB 00:00 2024-04-25T10:42:06.222 INFO:teuthology.orchestra.run.smithi019.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-25T10:42:06.236 INFO:teuthology.orchestra.run.smithi169.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 273 kB/s | 18 kB 00:00 2024-04-25T10:42:06.256 INFO:teuthology.orchestra.run.smithi019.stdout:(107/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-04-25T10:42:06.282 INFO:teuthology.orchestra.run.smithi019.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-04-25T10:42:06.323 INFO:teuthology.orchestra.run.smithi019.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-25T10:42:06.370 INFO:teuthology.orchestra.run.smithi169.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 483 kB/s | 80 kB 00:00 2024-04-25T10:42:06.399 INFO:teuthology.orchestra.run.smithi019.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-25T10:42:06.422 INFO:teuthology.orchestra.run.smithi112.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-04-25T10:42:06.424 INFO:teuthology.orchestra.run.smithi019.stdout:(111/152): liboath-2.6.7-2.el9.x86_64.rpm 290 kB/s | 49 kB 00:00 2024-04-25T10:42:06.448 INFO:teuthology.orchestra.run.smithi112.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 10 kB/s | 26 kB 00:02 2024-04-25T10:42:06.449 INFO:teuthology.orchestra.run.smithi019.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T10:42:06.462 INFO:teuthology.orchestra.run.smithi169.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-25T10:42:06.473 INFO:teuthology.orchestra.run.smithi112.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 19 kB/s | 49 kB 00:02 2024-04-25T10:42:06.498 INFO:teuthology.orchestra.run.smithi112.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-04-25T10:42:06.520 INFO:teuthology.orchestra.run.smithi169.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-25T10:42:06.582 INFO:teuthology.orchestra.run.smithi019.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-25T10:42:06.587 INFO:teuthology.orchestra.run.smithi169.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-25T10:42:06.590 INFO:teuthology.orchestra.run.smithi112.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T10:42:06.625 INFO:teuthology.orchestra.run.smithi019.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-04-25T10:42:06.654 INFO:teuthology.orchestra.run.smithi169.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-25T10:42:06.658 INFO:teuthology.orchestra.run.smithi019.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 422 kB/s | 32 kB 00:00 2024-04-25T10:42:06.680 INFO:teuthology.orchestra.run.smithi169.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T10:42:06.683 INFO:teuthology.orchestra.run.smithi019.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-25T10:42:06.709 INFO:teuthology.orchestra.run.smithi019.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T10:42:06.713 INFO:teuthology.orchestra.run.smithi169.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:42:06.734 INFO:teuthology.orchestra.run.smithi019.stdout:(118/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-25T10:42:06.738 INFO:teuthology.orchestra.run.smithi169.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T10:42:06.759 INFO:teuthology.orchestra.run.smithi019.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-25T10:42:06.781 INFO:teuthology.orchestra.run.smithi169.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-25T10:42:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 706 kB/s | 35 kB 00:00 2024-04-25T10:42:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-04-25T10:42:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T10:42:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T10:42:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-25T10:42:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 525 kB/s | 18 kB 00:00 2024-04-25T10:42:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 694 kB/s | 23 kB 00:00 2024-04-25T10:42:06.937 INFO:teuthology.orchestra.run.smithi019.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 580 kB/s | 19 kB 00:00 2024-04-25T10:42:06.962 INFO:teuthology.orchestra.run.smithi019.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 466 kB/s | 20 kB 00:00 2024-04-25T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-04-25T10:42:07.020 INFO:teuthology.orchestra.run.smithi019.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T10:42:07.046 INFO:teuthology.orchestra.run.smithi019.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-04-25T10:42:07.071 INFO:teuthology.orchestra.run.smithi019.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T10:42:07.096 INFO:teuthology.orchestra.run.smithi019.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T10:42:07.130 INFO:teuthology.orchestra.run.smithi019.stdout:(133/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-04-25T10:42:07.155 INFO:teuthology.orchestra.run.smithi019.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-25T10:42:07.180 INFO:teuthology.orchestra.run.smithi019.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T10:42:07.206 INFO:teuthology.orchestra.run.smithi019.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-25T10:42:07.231 INFO:teuthology.orchestra.run.smithi019.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T10:42:07.232 INFO:teuthology.orchestra.run.smithi112.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 747 kB/s | 548 kB 00:00 2024-04-25T10:42:07.247 INFO:teuthology.orchestra.run.smithi169.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.3 MB/s | 689 kB 00:00 2024-04-25T10:42:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-25T10:42:07.257 INFO:teuthology.orchestra.run.smithi112.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 65 kB/s | 43 kB 00:00 2024-04-25T10:42:07.281 INFO:teuthology.orchestra.run.smithi019.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T10:42:07.282 INFO:teuthology.orchestra.run.smithi112.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 107 kB/s | 32 kB 00:00 2024-04-25T10:42:07.306 INFO:teuthology.orchestra.run.smithi169.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-04-25T10:42:07.307 INFO:teuthology.orchestra.run.smithi019.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-25T10:42:07.308 INFO:teuthology.orchestra.run.smithi112.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-25T10:42:07.332 INFO:teuthology.orchestra.run.smithi019.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T10:42:07.357 INFO:teuthology.orchestra.run.smithi169.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 982 kB/s | 107 kB 00:00 2024-04-25T10:42:07.357 INFO:teuthology.orchestra.run.smithi019.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-25T10:42:07.382 INFO:teuthology.orchestra.run.smithi169.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-25T10:42:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T10:42:07.391 INFO:teuthology.orchestra.run.smithi112.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T10:42:07.408 INFO:teuthology.orchestra.run.smithi019.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T10:42:07.433 INFO:teuthology.orchestra.run.smithi019.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-04-25T10:42:07.440 INFO:teuthology.orchestra.run.smithi169.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-04-25T10:42:07.458 INFO:teuthology.orchestra.run.smithi019.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-25T10:42:07.483 INFO:teuthology.orchestra.run.smithi019.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T10:42:07.542 INFO:teuthology.orchestra.run.smithi019.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T10:42:07.557 INFO:teuthology.orchestra.run.smithi169.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-04-25T10:42:07.575 INFO:teuthology.orchestra.run.smithi112.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-04-25T10:42:07.608 INFO:teuthology.orchestra.run.smithi169.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-25T10:42:07.658 INFO:teuthology.orchestra.run.smithi169.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-25T10:42:07.692 INFO:teuthology.orchestra.run.smithi112.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 790 kB/s | 303 kB 00:00 2024-04-25T10:42:07.717 INFO:teuthology.orchestra.run.smithi112.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 108 kB/s | 35 kB 00:00 2024-04-25T10:42:07.750 INFO:teuthology.orchestra.run.smithi169.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-25T10:42:07.801 INFO:teuthology.orchestra.run.smithi112.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 912 kB/s | 206 kB 00:00 2024-04-25T10:42:07.808 INFO:teuthology.orchestra.run.smithi169.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-04-25T10:42:07.884 INFO:teuthology.orchestra.run.smithi019.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.0 MB/s | 1.3 MB 00:00 2024-04-25T10:42:08.051 INFO:teuthology.orchestra.run.smithi169.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2024-04-25T10:42:08.051 INFO:teuthology.orchestra.run.smithi019.stdout:(150/152): librados2-19.0.0-3291.g994e2f20.el9. 5.6 MB/s | 3.3 MB 00:00 2024-04-25T10:42:08.101 INFO:teuthology.orchestra.run.smithi019.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-04-25T10:42:08.142 INFO:teuthology.orchestra.run.smithi169.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.3 MB/s | 3.0 MB 00:00 2024-04-25T10:42:08.201 INFO:teuthology.orchestra.run.smithi169.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-25T10:42:08.260 INFO:teuthology.orchestra.run.smithi169.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 2.7 MB/s | 582 kB 00:00 2024-04-25T10:42:08.285 INFO:teuthology.orchestra.run.smithi169.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-04-25T10:42:08.309 INFO:teuthology.orchestra.run.smithi019.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 2.4 MB 00:00 2024-04-25T10:42:08.312 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:42:08.312 INFO:teuthology.orchestra.run.smithi019.stdout:Total 12 MB/s | 214 MB 00:17 2024-04-25T10:42:08.344 INFO:teuthology.orchestra.run.smithi169.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T10:42:08.377 INFO:teuthology.orchestra.run.smithi169.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-25T10:42:08.403 INFO:teuthology.orchestra.run.smithi169.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-04-25T10:42:08.593 INFO:teuthology.orchestra.run.smithi112.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-25T10:42:08.651 INFO:teuthology.orchestra.run.smithi112.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 155 kB/s | 144 kB 00:00 2024-04-25T10:42:08.676 INFO:teuthology.orchestra.run.smithi112.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 12 kB/s | 11 kB 00:00 2024-04-25T10:42:08.702 INFO:teuthology.orchestra.run.smithi112.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T10:42:08.727 INFO:teuthology.orchestra.run.smithi112.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-25T10:42:08.761 INFO:teuthology.orchestra.run.smithi112.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-04-25T10:42:08.786 INFO:teuthology.orchestra.run.smithi112.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-25T10:42:09.036 INFO:teuthology.orchestra.run.smithi169.stdout:(66/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 7.1 MB/s | 43 MB 00:06 2024-04-25T10:42:09.103 INFO:teuthology.orchestra.run.smithi112.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 2.7 MB/s | 1.0 MB 00:00 2024-04-25T10:42:09.103 INFO:teuthology.orchestra.run.smithi169.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-04-25T10:42:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 126 kB/s | 46 kB 00:00 2024-04-25T10:42:09.153 INFO:teuthology.orchestra.run.smithi112.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 215 kB/s | 79 kB 00:00 2024-04-25T10:42:09.170 INFO:teuthology.orchestra.run.smithi169.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.0 MB/s | 4.7 MB 00:00 2024-04-25T10:42:09.178 INFO:teuthology.orchestra.run.smithi112.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 769 kB/s | 58 kB 00:00 2024-04-25T10:42:09.220 INFO:teuthology.orchestra.run.smithi169.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-25T10:42:09.254 INFO:teuthology.orchestra.run.smithi169.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.9 MB/s | 911 kB 00:00 2024-04-25T10:42:09.287 INFO:teuthology.orchestra.run.smithi112.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-25T10:42:09.312 INFO:teuthology.orchestra.run.smithi112.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 104 kB/s | 16 kB 00:00 2024-04-25T10:42:09.329 INFO:teuthology.orchestra.run.smithi112.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 598 kB/s | 90 kB 00:00 2024-04-25T10:42:09.355 INFO:teuthology.orchestra.run.smithi112.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-25T10:42:09.362 INFO:teuthology.orchestra.run.smithi169.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-25T10:42:09.387 INFO:teuthology.orchestra.run.smithi169.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-25T10:42:09.388 INFO:teuthology.orchestra.run.smithi112.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-04-25T10:42:09.413 INFO:teuthology.orchestra.run.smithi169.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-25T10:42:09.439 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T10:42:09.446 INFO:teuthology.orchestra.run.smithi112.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-25T10:42:09.463 INFO:teuthology.orchestra.run.smithi169.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-25T10:42:09.481 INFO:teuthology.orchestra.run.smithi112.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 470 kB/s | 59 kB 00:00 2024-04-25T10:42:09.506 INFO:teuthology.orchestra.run.smithi112.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-04-25T10:42:09.513 INFO:teuthology.orchestra.run.smithi169.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T10:42:09.531 INFO:teuthology.orchestra.run.smithi112.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-04-25T10:42:09.564 INFO:teuthology.orchestra.run.smithi169.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T10:42:09.597 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T10:42:09.597 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T10:42:09.606 INFO:teuthology.orchestra.run.smithi112.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-25T10:42:09.614 INFO:teuthology.orchestra.run.smithi169.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T10:42:09.623 INFO:teuthology.orchestra.run.smithi112.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 768 kB/s | 90 kB 00:00 2024-04-25T10:42:09.664 INFO:teuthology.orchestra.run.smithi169.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T10:42:09.749 INFO:teuthology.orchestra.run.smithi112.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-25T10:42:09.765 INFO:teuthology.orchestra.run.smithi169.stdout:(79/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-25T10:42:09.774 INFO:teuthology.orchestra.run.smithi112.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 120 kB/s | 20 kB 00:00 2024-04-25T10:42:09.790 INFO:teuthology.orchestra.run.smithi169.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 277 kB/s | 35 kB 00:00 2024-04-25T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 917 kB/s | 192 kB 00:00 2024-04-25T10:42:09.890 INFO:teuthology.orchestra.run.smithi169.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-25T10:42:09.924 INFO:teuthology.orchestra.run.smithi169.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-25T10:42:09.949 INFO:teuthology.orchestra.run.smithi169.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T10:42:09.983 INFO:teuthology.orchestra.run.smithi169.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T10:42:10.008 INFO:teuthology.orchestra.run.smithi169.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 725 kB/s | 42 kB 00:00 2024-04-25T10:42:10.041 INFO:teuthology.orchestra.run.smithi169.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:42:10.067 INFO:teuthology.orchestra.run.smithi169.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T10:42:10.092 INFO:teuthology.orchestra.run.smithi169.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T10:42:10.118 INFO:teuthology.orchestra.run.smithi169.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T10:42:10.143 INFO:teuthology.orchestra.run.smithi169.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T10:42:10.158 INFO:teuthology.orchestra.run.smithi112.stdout:(148/152): librados2-19.0.0-3291.g994e2f20.el9. 8.7 MB/s | 3.3 MB 00:00 2024-04-25T10:42:10.208 INFO:teuthology.orchestra.run.smithi112.stdout:(149/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-04-25T10:42:10.226 INFO:teuthology.orchestra.run.smithi169.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-25T10:42:10.251 INFO:teuthology.orchestra.run.smithi169.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-04-25T10:42:10.277 INFO:teuthology.orchestra.run.smithi169.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-04-25T10:42:10.302 INFO:teuthology.orchestra.run.smithi169.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T10:42:10.317 INFO:teuthology.orchestra.run.smithi112.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 1.3 MB 00:00 2024-04-25T10:42:10.327 INFO:teuthology.orchestra.run.smithi169.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-04-25T10:42:10.353 INFO:teuthology.orchestra.run.smithi169.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T10:42:10.358 INFO:teuthology.orchestra.run.smithi112.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-04-25T10:42:10.378 INFO:teuthology.orchestra.run.smithi169.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T10:42:10.403 INFO:teuthology.orchestra.run.smithi169.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T10:42:10.428 INFO:teuthology.orchestra.run.smithi169.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T10:42:10.433 INFO:teuthology.orchestra.run.smithi112.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T10:42:10.438 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:42:10.438 INFO:teuthology.orchestra.run.smithi112.stdout:Total 11 MB/s | 214 MB 00:20 2024-04-25T10:42:10.454 INFO:teuthology.orchestra.run.smithi169.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T10:42:10.479 INFO:teuthology.orchestra.run.smithi169.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-25T10:42:10.504 INFO:teuthology.orchestra.run.smithi169.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-25T10:42:10.538 INFO:teuthology.orchestra.run.smithi169.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T10:42:10.938 INFO:teuthology.orchestra.run.smithi169.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-25T10:42:11.089 INFO:teuthology.orchestra.run.smithi169.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 561 kB/s | 309 kB 00:00 2024-04-25T10:42:11.429 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T10:42:11.429 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T10:42:11.501 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T10:42:11.606 INFO:teuthology.orchestra.run.smithi169.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 498 kB/s | 548 kB 00:01 2024-04-25T10:42:11.631 INFO:teuthology.orchestra.run.smithi169.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 28 kB/s | 19 kB 00:00 2024-04-25T10:42:11.642 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T10:42:11.642 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T10:42:12.135 INFO:teuthology.orchestra.run.smithi169.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 49 kB/s | 26 kB 00:00 2024-04-25T10:42:12.165 INFO:teuthology.orchestra.run.smithi169.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 91 kB/s | 49 kB 00:00 2024-04-25T10:42:12.724 INFO:teuthology.orchestra.run.smithi169.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-04-25T10:42:12.740 INFO:teuthology.orchestra.run.smithi169.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 111 kB/s | 67 kB 00:00 2024-04-25T10:42:12.765 INFO:teuthology.orchestra.run.smithi169.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 252 kB/s | 151 kB 00:00 2024-04-25T10:42:12.907 INFO:teuthology.orchestra.run.smithi169.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T10:42:12.966 INFO:teuthology.orchestra.run.smithi169.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-25T10:42:12.991 INFO:teuthology.orchestra.run.smithi169.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 193 kB/s | 43 kB 00:00 2024-04-25T10:42:13.016 INFO:teuthology.orchestra.run.smithi169.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 298 kB/s | 32 kB 00:00 2024-04-25T10:42:13.042 INFO:teuthology.orchestra.run.smithi169.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-25T10:42:13.109 INFO:teuthology.orchestra.run.smithi169.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T10:42:13.142 INFO:teuthology.orchestra.run.smithi169.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T10:42:13.176 INFO:teuthology.orchestra.run.smithi169.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T10:42:13.201 INFO:teuthology.orchestra.run.smithi169.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 386 kB/s | 35 kB 00:00 2024-04-25T10:42:13.226 INFO:teuthology.orchestra.run.smithi169.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T10:42:13.365 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T10:42:13.365 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T10:42:13.443 INFO:teuthology.orchestra.run.smithi169.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-04-25T10:42:13.469 INFO:teuthology.orchestra.run.smithi169.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 541 kB/s | 144 kB 00:00 2024-04-25T10:42:13.494 INFO:teuthology.orchestra.run.smithi169.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 40 kB/s | 11 kB 00:00 2024-04-25T10:42:13.519 INFO:teuthology.orchestra.run.smithi169.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-25T10:42:13.569 INFO:teuthology.orchestra.run.smithi169.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-25T10:42:13.594 INFO:teuthology.orchestra.run.smithi169.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-25T10:42:13.620 INFO:teuthology.orchestra.run.smithi169.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-25T10:42:13.753 INFO:teuthology.orchestra.run.smithi169.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-04-25T10:42:13.779 INFO:teuthology.orchestra.run.smithi169.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2024-04-25T10:42:13.795 INFO:teuthology.orchestra.run.smithi169.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 449 kB/s | 79 kB 00:00 2024-04-25T10:42:13.829 INFO:teuthology.orchestra.run.smithi169.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-04-25T10:42:13.879 INFO:teuthology.orchestra.run.smithi169.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-25T10:42:13.904 INFO:teuthology.orchestra.run.smithi169.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-25T10:42:13.930 INFO:teuthology.orchestra.run.smithi169.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-04-25T10:42:13.955 INFO:teuthology.orchestra.run.smithi169.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T10:42:14.005 INFO:teuthology.orchestra.run.smithi169.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-04-25T10:42:14.031 INFO:teuthology.orchestra.run.smithi169.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T10:42:14.047 INFO:teuthology.orchestra.run.smithi169.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-25T10:42:14.081 INFO:teuthology.orchestra.run.smithi169.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-25T10:42:14.115 INFO:teuthology.orchestra.run.smithi169.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 760 kB/s | 51 kB 00:00 2024-04-25T10:42:14.140 INFO:teuthology.orchestra.run.smithi169.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T10:42:14.165 INFO:teuthology.orchestra.run.smithi169.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-25T10:42:14.207 INFO:teuthology.orchestra.run.smithi169.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-25T10:42:14.224 INFO:teuthology.orchestra.run.smithi169.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T10:42:14.249 INFO:teuthology.orchestra.run.smithi169.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T10:42:14.400 INFO:teuthology.orchestra.run.smithi169.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-04-25T10:42:14.675 INFO:teuthology.orchestra.run.smithi169.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-04-25T10:42:14.825 INFO:teuthology.orchestra.run.smithi169.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T10:42:14.950 INFO:teuthology.orchestra.run.smithi169.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-04-25T10:42:15.042 INFO:teuthology.orchestra.run.smithi169.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 4.1 MB/s | 3.3 MB 00:00 2024-04-25T10:42:15.047 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:42:15.047 INFO:teuthology.orchestra.run.smithi169.stdout:Total 9.1 MB/s | 214 MB 00:23 2024-04-25T10:42:16.130 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-25T10:42:16.271 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-25T10:42:16.271 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-25T10:42:17.861 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T10:42:17.964 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:42:17.986 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-25T10:42:17.986 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-25T10:42:18.011 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:42:18.065 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:42:18.115 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:42:18.140 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:42:18.192 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:42:18.598 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:42:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:42:18.786 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:42:18.817 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:42:18.893 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:42:18.921 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:42:19.078 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:42:19.127 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:42:19.154 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:42:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:42:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:42:19.385 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:42:19.406 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:42:19.428 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:42:19.473 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T10:42:19.562 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:42:19.600 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:42:19.606 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:42:19.639 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:42:19.654 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:42:19.688 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:42:19.714 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:42:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:42:19.756 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:42:19.846 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:42:19.880 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:42:19.916 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:42:19.992 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:42:20.031 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:42:20.075 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:42:20.094 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:42:20.107 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:42:20.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:42:20.207 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:42:20.215 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:42:20.258 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:42:20.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:42:20.331 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:42:20.332 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:42:20.366 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:42:20.409 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:42:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:42:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:42:20.470 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:42:20.485 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:42:20.522 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:42:20.596 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:42:20.617 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:42:20.645 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:42:20.675 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:42:20.703 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:42:20.723 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:42:20.807 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:42:20.829 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:42:20.869 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:42:20.892 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:42:20.914 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:42:20.924 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:42:20.933 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:42:21.139 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:42:21.172 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:42:21.279 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:42:21.374 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:42:21.412 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:42:21.447 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:42:21.483 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:42:21.503 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:42:21.538 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:42:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:42:21.581 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:42:21.605 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:42:21.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:42:21.642 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:42:21.648 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:42:21.686 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:42:21.709 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:42:21.736 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:42:21.763 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:42:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:42:21.838 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:42:21.900 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:42:21.913 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:42:21.949 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:42:21.955 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:42:21.992 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:42:22.012 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:42:22.021 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:42:22.085 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:42:22.089 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:42:22.122 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:42:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:42:22.159 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:42:22.206 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:42:22.209 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:42:22.242 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:42:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:42:22.327 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:42:22.340 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:42:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:42:22.473 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:42:22.541 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:42:22.571 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:42:22.613 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:42:22.719 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:42:22.754 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:42:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:42:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:42:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:42:22.906 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:42:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:42:23.000 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:42:23.038 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:42:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:42:23.080 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:42:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:42:23.119 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:42:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:42:23.160 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:42:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:42:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:42:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:42:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:42:23.327 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:42:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:42:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:42:23.405 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:42:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:42:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:42:23.563 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:42:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:42:23.591 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:42:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:42:23.638 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:42:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:42:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:42:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:42:23.759 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:42:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-25T10:42:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:42:23.891 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:42:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:42:23.939 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:42:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:42:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:42:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:42:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:42:24.105 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:42:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:42:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:42:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:42:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:42:24.272 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:42:24.297 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:42:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:42:24.341 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:42:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:42:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:42:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:42:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:42:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:42:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:42:24.641 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:42:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:42:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:42:24.683 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:42:24.684 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:42:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:42:24.718 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:42:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:42:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:42:24.806 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:42:24.823 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:42:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:42:24.885 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:42:24.966 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:42:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:42:24.983 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:42:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:42:25.028 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:42:25.057 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:42:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:42:25.096 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:42:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:42:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:42:25.214 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:42:25.242 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:42:25.250 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:42:25.301 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:42:25.329 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:42:25.331 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:42:25.432 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:42:25.454 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:42:25.517 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:42:25.539 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:42:25.564 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:42:25.764 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:42:25.797 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:42:25.905 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:42:25.918 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:42:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:42:25.993 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:42:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:42:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:42:26.029 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:42:26.079 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:42:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:42:26.129 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:42:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:42:26.154 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:42:26.188 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:42:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:42:26.232 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:42:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:42:26.264 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:42:26.301 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:42:26.370 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:42:26.423 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:42:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:42:26.451 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:42:26.453 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:42:26.489 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:42:26.528 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:42:26.566 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:42:26.600 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:42:26.643 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:42:26.672 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:42:26.744 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:42:26.795 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:42:26.870 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:42:27.006 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:42:27.065 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:42:27.160 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:42:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:42:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:42:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:42:27.540 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:42:27.573 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:42:27.596 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:42:27.677 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:42:27.705 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:42:27.748 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:42:27.911 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:42:27.953 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:42:27.992 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:42:28.064 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:42:28.119 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:42:28.193 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:42:28.194 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:42:28.231 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:42:28.268 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:42:28.291 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:42:28.315 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:42:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:42:28.351 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:42:28.394 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:42:28.445 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:42:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:42:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:42:28.583 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:42:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:42:28.633 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:42:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:42:28.672 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:42:28.713 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:42:28.809 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:42:28.843 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:42:28.846 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:42:28.872 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:42:28.895 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:42:28.929 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:42:28.935 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:42:28.953 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:42:28.965 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:42:28.972 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:42:29.014 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:42:29.030 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:42:29.114 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:42:29.144 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:42:29.157 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:42:29.180 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:42:29.194 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:42:29.222 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:42:29.257 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:42:29.296 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:42:29.328 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:42:29.357 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:42:29.383 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:42:29.419 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:42:29.458 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:42:29.492 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:42:29.581 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:42:29.623 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:42:29.655 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:42:29.675 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:42:29.702 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:42:29.709 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:42:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:42:29.809 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:42:29.810 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:42:29.881 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:42:30.619 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:42:30.669 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:42:30.703 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:42:30.804 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:42:30.862 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:42:30.950 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:42:30.997 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:42:31.176 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:42:31.203 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:42:31.281 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:42:31.291 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:31.329 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:31.919 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:31.938 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:42:31.958 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:42:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:42:32.048 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:42:32.048 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:42:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:32.053 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:42:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:32.075 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:42:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:32.837 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:32.877 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:42:32.949 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:42:32.949 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:42:32.950 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:42:32.950 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:32.970 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:42:33.298 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:42:33.399 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:42:33.427 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:42:33.460 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:42:33.780 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:42:33.823 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:42:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:42:33.944 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:42:33.980 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:42:34.470 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:42:34.713 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:42:35.005 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:42:35.035 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:42:37.273 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:37.307 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:37.864 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:42:37.902 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:42:37.994 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:42:37.995 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:42:37.995 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:42:37.995 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:38.028 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:42:44.097 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:42:44.097 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-04-25T10:42:44.097 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-04-25T10:42:44.097 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-04-25T10:42:44.097 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-04-25T10:42:44.097 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-04-25T10:42:44.097 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-04-25T10:42:44.098 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-04-25T10:42:44.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:44.277 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:42:44.290 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-25T10:42:44.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:44.293 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:42:44.293 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:44.293 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:42:44.293 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:44.466 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:42:44.480 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:42:44.480 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:44.481 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:42:44.481 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:45.931 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:42:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:42:45.958 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:45.958 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:42:45.958 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:42:45.958 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:42:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:46.069 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:42:46.094 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:42:46.094 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:46.094 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:42:46.094 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:42:46.094 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:42:46.094 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:46.190 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:42:46.213 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:42:46.213 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:46.213 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:42:46.214 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:42:46.214 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:42:46.214 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:46.319 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:42:46.341 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:42:46.341 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:46.341 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:42:46.341 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:42:46.341 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:42:46.341 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:46.698 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:42:46.722 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:42:46.722 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:46.722 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:42:46.722 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:42:46.722 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:42:46.722 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:46.795 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:42:46.817 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:42:46.817 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:46.817 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:42:46.817 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:42:46.817 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:42:46.817 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:46.838 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:42:46.895 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:42:46.917 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:42:46.973 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:42:46.977 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:42:47.029 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:42:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:42:47.212 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:42:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:42:47.263 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:42:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:42:47.341 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:42:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:42:47.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:42:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:42:47.567 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:42:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:42:47.604 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:42:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:42:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:42:47.645 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:42:47.673 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:42:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:42:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:42:48.150 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:42:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:42:48.180 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:42:48.212 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:42:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:42:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:42:48.274 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:42:48.311 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:42:49.350 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:42:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:42:49.388 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:42:49.390 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-25T10:42:49.590 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-25T10:42:49.591 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:49.783 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:42:49.798 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:42:49.799 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:49.799 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:42:49.799 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:42:50.371 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:42:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:42:50.434 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:42:50.473 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:42:50.538 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:42:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:42:50.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:42:50.696 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:42:50.696 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:50.696 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:42:50.696 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:42:50.697 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:42:50.697 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:42:50.734 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:42:50.741 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:42:50.752 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:42:50.758 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:42:50.758 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:50.758 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:42:50.758 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:42:50.758 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:42:50.758 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:50.809 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:42:50.820 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:42:50.851 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:42:50.906 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:42:50.924 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:42:50.976 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:42:51.386 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:42:51.411 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:42:51.411 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:51.412 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:42:51.412 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:42:51.412 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:42:51.412 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:51.636 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:42:51.658 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:42:51.658 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:51.658 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:42:51.659 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:42:51.659 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:42:51.659 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:51.842 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:42:51.869 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:42:51.891 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:42:51.891 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:51.891 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:42:51.891 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:42:51.891 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:42:51.891 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:51.936 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:42:51.957 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:42:51.957 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:51.958 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:42:51.958 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:51.962 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:42:51.996 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:42:52.019 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:42:52.019 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:52.019 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:42:52.019 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:42:52.020 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:42:52.020 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:52.028 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:42:52.050 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:42:52.050 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:52.050 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:42:52.051 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:42:52.051 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:42:52.051 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:52.063 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:42:52.086 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:42:52.086 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:52.086 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:42:52.086 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:52.120 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:42:52.138 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:42:52.143 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:42:52.143 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:52.143 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:42:52.143 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:42:52.143 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:42:52.143 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:52.162 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:42:52.162 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:52.162 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:42:52.162 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:42:52.162 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:42:52.163 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:52.242 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:42:52.299 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:42:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:42:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:42:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:42:52.384 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:52.385 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:42:52.385 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:42:52.385 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:42:52.385 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:42:52.489 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:42:52.513 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:42:52.513 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:52.513 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:42:52.513 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:42:52.513 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:42:52.513 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:42:52.542 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:42:52.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:42:52.696 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:42:52.797 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:42:52.937 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:42:52.980 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:42:53.011 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:42:53.039 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:42:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:42:53.421 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:42:53.453 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:42:53.515 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:42:53.553 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:42:54.604 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:42:54.646 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:42:55.618 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:42:55.646 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:42:55.749 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:42:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:42:55.972 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:42:55.972 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:55.972 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:42:55.972 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:42:55.972 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:42:55.972 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:56.016 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:42:56.027 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:42:56.126 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:42:56.173 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:42:56.675 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:42:56.716 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:42:56.756 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:42:56.788 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:42:56.816 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:42:56.997 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:42:57.035 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:42:57.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:42:57.075 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:42:57.076 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:42:57.112 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:42:57.116 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:42:57.143 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:42:57.150 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:42:57.166 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:42:57.166 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:57.166 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:42:57.167 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:42:57.167 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:42:57.167 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:57.178 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:42:57.211 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:42:57.233 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:42:57.233 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:57.233 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:42:57.233 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:57.278 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:42:57.300 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:42:57.301 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:57.301 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:42:57.301 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:42:57.301 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:42:57.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:57.377 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:42:57.415 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:42:57.445 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:42:57.612 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:42:57.634 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:42:57.635 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:42:57.635 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:42:57.635 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:42:57.635 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:42:57.635 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:42:57.705 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:42:57.748 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:42:57.767 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:42:58.005 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:42:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:42:58.063 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:42:58.589 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:42:58.678 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:42:58.719 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:42:58.801 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:42:58.832 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:42:58.832 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:42:58.916 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:42:58.949 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:42:58.949 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:42:58.951 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:42:59.038 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:42:59.048 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:42:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:42:59.106 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:42:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:42:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:42:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:42:59.295 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:42:59.328 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:42:59.328 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:42:59.406 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:42:59.479 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:01.933 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:43:01.974 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:43:02.023 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:43:02.055 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:43:02.082 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:43:02.271 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:43:02.320 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:43:02.350 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:43:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:43:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:43:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:43:03.894 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:43:04.002 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:43:04.044 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:43:04.127 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:43:04.166 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:43:04.166 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:43:04.258 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:43:04.291 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:43:04.292 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:43:04.385 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:43:04.456 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:43:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:43:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:43:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:43:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:43:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:43:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:43:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:43:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:43:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:43:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:43:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:43:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:43:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:43:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:43:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:43:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:43:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:43:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:43:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:43:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:43:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:43:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:43:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:43:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:43:07.154 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-25T10:43:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:43:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:43:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:43:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:43:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:43:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:43:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:43:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:43:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:43:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:43:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:43:07.166 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:43:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:43:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.362 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:43:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:43:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:43:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:43:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:43:07.367 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:43:07.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.369 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:43:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:43:07.371 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T10:43:07.553 DEBUG:teuthology.parallel:result is None 2024-04-25T10:43:07.777 DEBUG:teuthology.parallel:result is None 2024-04-25T10:43:10.628 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:10.628 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:43:10.628 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:43:10.628 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:43:10.628 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:43:10.628 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:43:10.629 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:43:10.630 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:43:10.642 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:43:10.642 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:43:10.642 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:43:10.642 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:43:10.642 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:43:10.642 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:43:10.643 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:43:10.644 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:43:10.645 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:43:10.646 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:43:10.647 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:43:10.648 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:43:10.649 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:43:10.650 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:43:10.651 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:43:10.652 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:43:12.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:43:12.764 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.765 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:43:12.766 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:43:12.767 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:43:12.768 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:43:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:43:12.770 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:43:12.771 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:43:12.772 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:43:12.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:43:12.774 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:43:12.775 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T10:43:13.127 DEBUG:teuthology.parallel:result is None 2024-04-25T10:43:13.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:43:13.344 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:43:13.371 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:43:13.372 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:43:13.372 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:43:13.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:43:13.507 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:43:13.533 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:43:13.533 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:43:13.533 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:43:13.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:43:13.675 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:43:13.696 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:43:13.697 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:43:13.697 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:43:13.699 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:43:13.699 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:43:13.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:43:13.731 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:43:13.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:43:13.761 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:43:13.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:43:13.788 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:43:13.788 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:43:13.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:43:13.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:43:13.888 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:43:13.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:43:13.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:43:13.984 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:43:13.984 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:43:14.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:43:14.073 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:43:14.073 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:43:14.073 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:43:14.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:43:14.171 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:43:14.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:43:14.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:43:14.267 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:43:14.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:43:14.292 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:43:14.357 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:43:14.357 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:43:14.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:43:14.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:43:14.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:43:14.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:43:14.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:43:14.552 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:43:14.552 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:43:14.577 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:43:14.641 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T10:43:14.745 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T10:43:14.746 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:43:14.746 INFO:tasks.cephadm:Cluster fsid is 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:43:14.746 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T10:43:14.746 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.112', 'mon.c': '172.21.15.169'} 2024-04-25T10:43:14.747 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-04-25T10:43:14.747 INFO:tasks.cephadm:First mgr is x 2024-04-25T10:43:14.747 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T10:43:14.747 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-04-25T10:43:14.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-04-25T10:43:14.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-04-25T10:43:14.830 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T10:43:14.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:43:15.046 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T10:43:15.182 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:43:15.183 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T10:43:15.184 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:15.744 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 10:43 /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:15.744 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:16.294 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 10:43 /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:16.294 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:17.011 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 10:43 /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:17.012 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:17.031 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:17.051 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T10:43:17.087 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T10:43:17.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T10:43:17.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T10:43:17.093 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T10:43:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:43:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:43:18.758 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:43:50.639 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:43:50.640 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:43:50.640 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T10:43:50.640 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-04-25T10:43:50.640 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T10:43:50.640 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-04-25T10:43:50.640 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:44:12.273 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T10:44:12.274 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:44:12.274 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T10:44:12.274 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-04-25T10:44:12.274 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T10:44:12.274 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T10:44:12.274 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T10:44:18.259 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:44:18.259 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:44:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T10:44:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-04-25T10:44:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T10:44:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:44:18.260 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:44:18.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-04-25T10:44:18.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-04-25T10:44:18.352 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-04-25T10:44:18.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-04-25T10:44:18.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-04-25T10:44:18.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-04-25T10:44:18.465 INFO:tasks.cephadm:Writing seed config... 2024-04-25T10:44:18.466 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T10:44:18.466 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T10:44:18.467 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T10:44:18.468 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T10:44:18.469 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T10:44:18.470 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T10:44:18.470 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T10:44:18.470 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T10:44:18.470 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:44:18.471 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T10:44:18.486 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9e90b6fe-02f0-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T10:44:18.486 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T10:44:18.529 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T10:44:18.573 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T10:44:18.573 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:44:18.769 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:44:18.769 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', '9e90b6fe-02f0-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-04-25T10:44:18.770 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T10:44:18.770 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-04-25T10:44:18.796 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T10:44:18.796 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-04-25T10:44:18.796 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-04-25T10:44:18.804 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T10:44:18.804 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T10:44:18.810 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T10:44:18.810 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T10:44:18.816 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-25T10:44:18.823 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-25T10:44:18.823 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-25T10:44:18.823 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-04-25T10:44:18.847 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T10:44:18.847 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T10:44:18.847 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-04-25T10:44:18.847 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-04-25T10:44:18.853 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T10:44:18.853 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T10:44:18.859 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T10:44:18.859 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T10:44:18.866 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-25T10:44:18.872 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-25T10:44:18.872 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-25T10:44:18.872 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-04-25T10:44:18.872 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:18.872 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140666890157216 on /run/cephadm/9e90b6fe-02f0-11ef-bc93-c7b262605968.lock 2024-04-25T10:44:18.873 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140666890157216 acquired on /run/cephadm/9e90b6fe-02f0-11ef-bc93-c7b262605968.lock 2024-04-25T10:44:18.873 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-04-25T10:44:18.873 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-04-25T10:44:18.873 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-25T10:44:18.877 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-04-25T10:44:18.877 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-04-25T10:44:18.879 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T10:44:18.879 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T10:44:18.879 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T10:44:18.882 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T10:44:18.882 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T10:44:18.882 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T10:44:18.883 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T10:44:18.883 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:44:18.883 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T10:44:18.883 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-25T10:44:18.883 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-25T10:44:18.884 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T10:44:18.884 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T10:44:18.884 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:44:19.388 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T10:44:19.388 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:44:19.388 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T10:44:19.388 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T10:44:19.388 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T10:44:19.388 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T10:44:19.388 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T10:44:20.342 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:44:20.342 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:44:20.342 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T10:44:21.313 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-04-25T10:44:21.313 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-04-25T10:44:22.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAFNCpmf5y7JRAAZXDDLI3ImXkBoHnE6XD4+A== 2024-04-25T10:44:23.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAGNCpmL6HfGxAAPraK1McO9nbYXzBhiXw3WQ== 2024-04-25T10:44:24.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAHNCpmtk0AGBAAyH9JImewUqfYVEyiQn6H3w== 2024-04-25T10:44:24.044 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:44:25.139 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-04-25T10:44:26.060 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-04-25T10:44:26.269 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T10:44:26.439 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T10:44:26.629 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968.target. 2024-04-25T10:44:26.629 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968.target. 2024-04-25T10:44:26.872 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a 2024-04-25T10:44:26.872 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T10:44:27.048 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968.target.wants/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@.service. 2024-04-25T10:44:27.560 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:44:27.560 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T10:44:27.567 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:44:27.567 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T10:44:27.567 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T10:44:27.567 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T10:44:27.567 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-04-25T10:44:27.567 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-04-25T10:44:28.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T10:44:28.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:28.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T10:44:28.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:28.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-04-25T10:44:28.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.574095s) 2024-04-25T10:44:28.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.016+0000 7fc14625d700 1 Processor -- start 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.016+0000 7fc14625d700 1 -- start start 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.016+0000 7fc14625d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc140109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.016+0000 7fc14625d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc14010a490 con 0x7fc140109a90 2024-04-25T10:44:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc140109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc140109ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47076/0 (socket says 172.21.15.19:47076) 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13f7fe700 1 -- 172.21.15.19:0/904997054 learned_addr learned my addr 172.21.15.19:0/904997054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13f7fe700 1 -- 172.21.15.19:0/904997054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14010ac60 con 0x7fc140109a90 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13f7fe700 1 --2- 172.21.15.19:0/904997054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc140109ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc1300098a0 tx=0x7fc130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef68977539c06e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13e7fc700 1 -- 172.21.15.19:0/904997054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130004160 con 0x7fc140109a90 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13e7fc700 1 -- 172.21.15.19:0/904997054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc130004300 con 0x7fc140109a90 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.017+0000 7fc13e7fc700 1 -- 172.21.15.19:0/904997054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1300036a0 con 0x7fc140109a90 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.018+0000 7fc14625d700 1 -- 172.21.15.19:0/904997054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 msgr2=0x7fc140109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.018+0000 7fc14625d700 1 --2- 172.21.15.19:0/904997054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc140109ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc1300098a0 tx=0x7fc130009bb0 comp rx=0 tx=0).stop 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.018+0000 7fc14625d700 1 -- 172.21.15.19:0/904997054 shutdown_connections 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.018+0000 7fc14625d700 1 --2- 172.21.15.19:0/904997054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc140109ea0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.018+0000 7fc14625d700 1 -- 172.21.15.19:0/904997054 >> 172.21.15.19:0/904997054 conn(0x7fc1401050f0 msgr2=0x7fc140107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:28.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.018+0000 7fc14625d700 1 -- 172.21.15.19:0/904997054 shutdown_connections 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.018+0000 7fc14625d700 1 -- 172.21.15.19:0/904997054 wait complete. 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.019+0000 7fc14625d700 1 Processor -- start 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.019+0000 7fc14625d700 1 -- start start 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.019+0000 7fc14625d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc14019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.019+0000 7fc14625d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc14010b190 con 0x7fc140109a90 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc14019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc14019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47092/0 (socket says 172.21.15.19:47092) 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13f7fe700 1 -- 172.21.15.19:0/858473001 learned_addr learned my addr 172.21.15.19:0/858473001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13f7fe700 1 -- 172.21.15.19:0/858473001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14019dae0 con 0x7fc140109a90 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13f7fe700 1 --2- 172.21.15.19:0/858473001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc14019d5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc130000c00 tx=0x7fc130004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13cff9700 1 -- 172.21.15.19:0/858473001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130003db0 con 0x7fc140109a90 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13cff9700 1 -- 172.21.15.19:0/858473001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc130011630 con 0x7fc140109a90 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13cff9700 1 -- 172.21.15.19:0/858473001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130011950 con 0x7fc140109a90 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14019dd70 con 0x7fc140109a90 2024-04-25T10:44:28.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401a0980 con 0x7fc140109a90 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13cff9700 1 -- 172.21.15.19:0/858473001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc130011af0 con 0x7fc140109a90 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.020+0000 7fc13cff9700 1 -- 172.21.15.19:0/858473001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc13001c450 con 0x7fc140109a90 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.021+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc140064cf0 con 0x7fc140109a90 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.024+0000 7fc13cff9700 1 -- 172.21.15.19:0/858473001 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc130032020 con 0x7fc140109a90 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.096+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc14002e810 con 0x7fc140109a90 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.097+0000 7fc13cff9700 1 -- 172.21.15.19:0/858473001 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc130004260 con 0x7fc140109a90 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.098+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 msgr2=0x7fc14019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.098+0000 7fc14625d700 1 --2- 172.21.15.19:0/858473001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc14019d5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc130000c00 tx=0x7fc130004750 comp rx=0 tx=0).stop 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.098+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 shutdown_connections 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.098+0000 7fc14625d700 1 --2- 172.21.15.19:0/858473001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc140109a90 0x7fc14019d5a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.098+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 >> 172.21.15.19:0/858473001 conn(0x7fc1401050f0 msgr2=0x7fc140193c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.098+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 shutdown_connections 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:28.098+0000 7fc14625d700 1 -- 172.21.15.19:0/858473001 wait complete. 2024-04-25T10:44:28.633 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-04-25T10:44:28.634 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T10:44:29.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:29 smithi019 ceph-mon[28835]: from='client.? 172.21.15.19:0/4022249525' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.044+0000 7ff0dd876700 1 Processor -- start 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.044+0000 7ff0dd876700 1 -- start start 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.044+0000 7ff0dd876700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d8108210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:29.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.045+0000 7ff0dd876700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8108800 con 0x7ff0d8107e00 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.045+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d8108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.045+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d8108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47108/0 (socket says 172.21.15.19:47108) 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.045+0000 7ff0d6ffd700 1 -- 172.21.15.19:0/2008425882 learned_addr learned my addr 172.21.15.19:0/2008425882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.045+0000 7ff0d6ffd700 1 -- 172.21.15.19:0/2008425882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d8108980 con 0x7ff0d8107e00 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.045+0000 7ff0d6ffd700 1 --2- 172.21.15.19:0/2008425882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d8108210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009e80 tx=0x7ff0c800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d69ad581441cb80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0d5ffb700 1 -- 172.21.15.19:0/2008425882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c8009200 con 0x7ff0d8107e00 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0d5ffb700 1 -- 172.21.15.19:0/2008425882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c8003a20 con 0x7ff0d8107e00 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0d5ffb700 1 -- 172.21.15.19:0/2008425882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c8003d40 con 0x7ff0d8107e00 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0dd876700 1 -- 172.21.15.19:0/2008425882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 msgr2=0x7ff0d8108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0dd876700 1 --2- 172.21.15.19:0/2008425882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d8108210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009e80 tx=0x7ff0c800f740 comp rx=0 tx=0).stop 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0dd876700 1 -- 172.21.15.19:0/2008425882 shutdown_connections 2024-04-25T10:44:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0dd876700 1 --2- 172.21.15.19:0/2008425882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d8108210 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0dd876700 1 -- 172.21.15.19:0/2008425882 >> 172.21.15.19:0/2008425882 conn(0x7ff0d81037a0 msgr2=0x7ff0d8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0dd876700 1 -- 172.21.15.19:0/2008425882 shutdown_connections 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.046+0000 7ff0dd876700 1 -- 172.21.15.19:0/2008425882 wait complete. 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.047+0000 7ff0dd876700 1 Processor -- start 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.047+0000 7ff0dd876700 1 -- start start 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.047+0000 7ff0dd876700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.047+0000 7ff0dd876700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8108ec0 con 0x7ff0d8107e00 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47112/0 (socket says 172.21.15.19:47112) 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0d6ffd700 1 -- 172.21.15.19:0/4022249525 learned_addr learned my addr 172.21.15.19:0/4022249525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0d6ffd700 1 -- 172.21.15.19:0/4022249525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d819b910 con 0x7ff0d8107e00 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0d6ffd700 1 --2- 172.21.15.19:0/4022249525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d819b3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8019040 tx=0x7ff0c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:29.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c8027070 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c8003d00 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c8021560 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d819bba0 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d819e7b0 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0c802a070 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.048+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff0c8014070 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.049+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d8064cf0 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.052+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff0c8031030 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.122+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff0d802e850 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.126+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7ff0c80217d0 con 0x7ff0d8107e00 2024-04-25T10:44:29.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 msgr2=0x7ff0d819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dc874700 1 -- 172.21.15.19:0/4022249525 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff0c8003a20 con 0x7ff0d8107e00 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dd876700 1 --2- 172.21.15.19:0/4022249525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d819b3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8019040 tx=0x7ff0c8004ad0 comp rx=0 tx=0).stop 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 shutdown_connections 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dd876700 1 --2- 172.21.15.19:0/4022249525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8107e00 0x7ff0d819b3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 >> 172.21.15.19:0/4022249525 conn(0x7ff0d81037a0 msgr2=0x7ff0d8191da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 shutdown_connections 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:29.127+0000 7ff0dd876700 1 -- 172.21.15.19:0/4022249525 wait complete. 2024-04-25T10:44:29.703 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-04-25T10:44:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.101+0000 7fda75861700 1 Processor -- start 2024-04-25T10:44:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.101+0000 7fda75861700 1 -- start start 2024-04-25T10:44:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.101+0000 7fda75861700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda70108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.101+0000 7fda75861700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda70109390 con 0x7fda70108990 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.101+0000 7fda6effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda70108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.101+0000 7fda6effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda70108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47116/0 (socket says 172.21.15.19:47116) 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.101+0000 7fda6effd700 1 -- 172.21.15.19:0/352278268 learned_addr learned my addr 172.21.15.19:0/352278268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.102+0000 7fda6effd700 1 -- 172.21.15.19:0/352278268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda70109b60 con 0x7fda70108990 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.102+0000 7fda6effd700 1 --2- 172.21.15.19:0/352278268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda70108da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fda600098a0 tx=0x7fda60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b21ecb8d41eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.102+0000 7fda6dffb700 1 -- 172.21.15.19:0/352278268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda60004160 con 0x7fda70108990 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.102+0000 7fda6dffb700 1 -- 172.21.15.19:0/352278268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fda60004300 con 0x7fda70108990 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.102+0000 7fda6dffb700 1 -- 172.21.15.19:0/352278268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda60011af0 con 0x7fda70108990 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.103+0000 7fda75861700 1 -- 172.21.15.19:0/352278268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 msgr2=0x7fda70108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.103+0000 7fda75861700 1 --2- 172.21.15.19:0/352278268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda70108da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fda600098a0 tx=0x7fda60009bb0 comp rx=0 tx=0).stop 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.103+0000 7fda75861700 1 -- 172.21.15.19:0/352278268 shutdown_connections 2024-04-25T10:44:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.103+0000 7fda75861700 1 --2- 172.21.15.19:0/352278268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda70108da0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.103+0000 7fda75861700 1 -- 172.21.15.19:0/352278268 >> 172.21.15.19:0/352278268 conn(0x7fda70103fd0 msgr2=0x7fda70106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.103+0000 7fda75861700 1 -- 172.21.15.19:0/352278268 shutdown_connections 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.103+0000 7fda75861700 1 -- 172.21.15.19:0/352278268 wait complete. 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda75861700 1 Processor -- start 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda75861700 1 -- start start 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda75861700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda7019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda75861700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda7010cf10 con 0x7fda70108990 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda6effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda7019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda6effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda7019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47128/0 (socket says 172.21.15.19:47128) 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda6effd700 1 -- 172.21.15.19:0/1467241623 learned_addr learned my addr 172.21.15.19:0/1467241623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.104+0000 7fda6effd700 1 -- 172.21.15.19:0/1467241623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7019e450 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda6effd700 1 --2- 172.21.15.19:0/1467241623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda7019df10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda60009fb0 tx=0x7fda60011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda7485f700 1 -- 172.21.15.19:0/1467241623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda600036a0 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda7485f700 1 -- 172.21.15.19:0/1467241623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fda60003cd0 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7019e6e0 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda7485f700 1 -- 172.21.15.19:0/1467241623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda6001a430 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda701a12f0 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda7485f700 1 -- 172.21.15.19:0/1467241623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda6001aa80 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.105+0000 7fda7485f700 1 -- 172.21.15.19:0/1467241623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda600193c0 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.106+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda70064cf0 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.108+0000 7fda7485f700 1 -- 172.21.15.19:0/1467241623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fda60024440 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.180+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fda700642a0 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.181+0000 7fda7485f700 1 -- 172.21.15.19:0/1467241623 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fda60024760 con 0x7fda70108990 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.182+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 msgr2=0x7fda7019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.182+0000 7fda75861700 1 --2- 172.21.15.19:0/1467241623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda7019df10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda60009fb0 tx=0x7fda60011890 comp rx=0 tx=0).stop 2024-04-25T10:44:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.182+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 shutdown_connections 2024-04-25T10:44:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.182+0000 7fda75861700 1 --2- 172.21.15.19:0/1467241623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda70108990 0x7fda7019df10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.182+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 >> 172.21.15.19:0/1467241623 conn(0x7fda70103fd0 msgr2=0x7fda701944c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.182+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 shutdown_connections 2024-04-25T10:44:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:30.182+0000 7fda75861700 1 -- 172.21.15.19:0/1467241623 wait complete. 2024-04-25T10:44:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-04-25T10:44:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:30 smithi019 ceph-mon[28835]: from='client.? 172.21.15.19:0/1467241623' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:44:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:30 smithi019 systemd[1]: Stopping Ceph mon.a for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:44:31.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a[28831]: 2024-04-25T10:44:30.947+0000 7f6d136e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:44:31.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a[28831]: 2024-04-25T10:44:30.947+0000 7f6d136e5700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T10:44:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:31 smithi019 podman[29081]: 2024-04-25 10:44:31.051550276 +0000 UTC m=+0.205073911 container died dbfa84c748db76868fb1bc53844fe80bf7132bcf720796d6590682510b87f03f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:44:31.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:31 smithi019 podman[29081]: 2024-04-25 10:44:31.448781136 +0000 UTC m=+0.602304780 container cleanup dbfa84c748db76868fb1bc53844fe80bf7132bcf720796d6590682510b87f03f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:44:31.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:31 smithi019 bash[29081]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a 2024-04-25T10:44:32.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:31 smithi019 podman[29095]: 2024-04-25 10:44:31.657957942 +0000 UTC m=+0.610051421 container remove dbfa84c748db76868fb1bc53844fe80bf7132bcf720796d6590682510b87f03f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:44:32.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:31 smithi019 systemd[1]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T10:44:32.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:31 smithi019 systemd[1]: Stopped Ceph mon.a for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:44:32.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:31 smithi019 systemd[1]: Starting Ceph mon.a for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:44:32.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 podman[29191]: 2024-04-25 10:44:32.083630216 +0000 UTC m=+0.095067158 container create ff48db461b118cb31cf002a543c99c86ae88d5be590aaf735aea7a09e3bed8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T10:44:32.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 podman[29191]: 2024-04-25 10:44:32.006774182 +0000 UTC m=+0.018211133 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:44:32.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 podman[29191]: 2024-04-25 10:44:32.277432997 +0000 UTC m=+0.288869939 container init ff48db461b118cb31cf002a543c99c86ae88d5be590aaf735aea7a09e3bed8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:44:32.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 podman[29191]: 2024-04-25 10:44:32.280304576 +0000 UTC m=+0.291741518 container start ff48db461b118cb31cf002a543c99c86ae88d5be590aaf735aea7a09e3bed8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T10:44:32.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: pidfile_write: ignore empty --pid-file 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: load: jerasure load: lrc 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: RocksDB version: 7.9.2 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Git sha 0 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: DB SUMMARY 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: DB Session ID: NH68ZV3V4BDINAPYCG4T 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: CURRENT file: CURRENT 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T10:44:32.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104880 ; 2024-04-25T10:44:32.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.error_if_exists: 0 2024-04-25T10:44:32.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.create_if_missing: 0 2024-04-25T10:44:32.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.paranoid_checks: 1 2024-04-25T10:44:32.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T10:44:32.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T10:44:32.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.env: 0x55bd73202500 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.info_log: 0x55bd759b9fc0 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.statistics: (nil) 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.use_fsync: 0 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_log_file_size: 0 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T10:44:32.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.allow_fallocate: 1 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.use_direct_reads: 0 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.db_log_dir: 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.wal_dir: 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T10:44:32.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.write_buffer_manager: 0x55bd7598a8c0 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T10:44:32.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.unordered_write: 0 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.row_cache: None 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.wal_filter: None 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T10:44:32.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.two_write_queues: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.wal_compression: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.atomic_flush: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.log_readahead_size: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T10:44:32.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_background_jobs: 2 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_background_compactions: -1 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_subcompactions: 1 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T10:44:32.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_open_files: -1 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_background_flushes: -1 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Compression algorithms supported: 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kZSTD supported: 0 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kXpressCompression supported: 0 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kZlibCompression supported: 1 2024-04-25T10:44:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kSnappyCompression supported: 1 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kLZ4Compression supported: 1 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: kBZip2Compression supported: 0 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.merge_operator: 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_filter: None 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T10:44:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bd75ac01a0) 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-04-25T10:44:32.373 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x55bd759bd1f0 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-04-25T10:44:32.374 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-04-25T10:44:32.375 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-04-25T10:44:32.375 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-04-25T10:44:32.375 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-04-25T10:44:32.375 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-04-25T10:44:32.375 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-04-25T10:44:32.375 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T10:44:32.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression: NoCompression 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.num_levels: 7 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T10:44:32.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T10:44:32.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T10:44:32.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T10:44:32.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.inplace_update_support: 0 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T10:44:32.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T10:44:32.381 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T10:44:32.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T10:44:32.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T10:44:32.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.bloom_locality: 0 2024-04-25T10:44:32.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.max_successive_merges: 0 2024-04-25T10:44:32.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T10:44:32.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.ttl: 2592000 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.enable_blob_files: false 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.min_blob_size: 0 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T10:44:32.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bbc2de12-feb9-4be3-ab5e-45790557ba94 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714041872320728, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714041872322078, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97910, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14131, "raw_average_key_size": 49, "raw_value_size": 90254, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714041872, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bbc2de12-feb9-4be3-ab5e-45790557ba94", "db_session_id": "NH68ZV3V4BDINAPYCG4T", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714041872322166, "job": 1, "event": "recovery_finished"} 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bd759a1c00 2024-04-25T10:44:32.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: DB pointer 0x55bd7a794000 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:44:32.385 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 99.54 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 99.54 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:44:32.386 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 27.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 27.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x55bd759bd1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T10:44:32.387 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???) e1 preinit fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???).mds e1 new map 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???).mds e1 print_map 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:44:32.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:44:32.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T10:44:32.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 bash[29191]: ff48db461b118cb31cf002a543c99c86ae88d5be590aaf735aea7a09e3bed8cb 2024-04-25T10:44:32.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 systemd[1]: Started Ceph mon.a for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:44:32.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T10:44:32.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: monmap epoch 1 2024-04-25T10:44:32.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:32.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: last_changed 2024-04-25T10:44:24.525437+0000 2024-04-25T10:44:32.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:44:32.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: min_mon_release 19 (squid) 2024-04-25T10:44:32.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: election_strategy: 1 2024-04-25T10:44:32.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:44:32.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: fsmap 2024-04-25T10:44:32.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T10:44:32.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:32 smithi019 ceph-mon[29209]: mgrmap e1: no daemons active 2024-04-25T10:44:33.467 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.829+0000 7f0d7d526700 1 Processor -- start 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.829+0000 7f0d7d526700 1 -- start start 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.829+0000 7f0d7d526700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d78108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.829+0000 7f0d7d526700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781093d0 con 0x7f0d781089d0 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d78108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d78108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37216/0 (socket says 172.21.15.19:37216) 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d76ffd700 1 -- 172.21.15.19:0/3190056038 learned_addr learned my addr 172.21.15.19:0/3190056038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d76ffd700 1 -- 172.21.15.19:0/3190056038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d78109ba0 con 0x7f0d781089d0 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d76ffd700 1 --2- 172.21.15.19:0/3190056038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d78108de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009e80 tx=0x7f0d7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143a7ae245c9c360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d75ffb700 1 -- 172.21.15.19:0/3190056038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d70017070 con 0x7f0d781089d0 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d75ffb700 1 -- 172.21.15.19:0/3190056038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d7000b8c0 con 0x7f0d781089d0 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.830+0000 7f0d75ffb700 1 -- 172.21.15.19:0/3190056038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d70004490 con 0x7f0d781089d0 2024-04-25T10:44:33.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.831+0000 7f0d7d526700 1 -- 172.21.15.19:0/3190056038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 msgr2=0x7f0d78108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.831+0000 7f0d7d526700 1 --2- 172.21.15.19:0/3190056038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d78108de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009e80 tx=0x7f0d7000b330 comp rx=0 tx=0).stop 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.831+0000 7f0d7d526700 1 -- 172.21.15.19:0/3190056038 shutdown_connections 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.831+0000 7f0d7d526700 1 --2- 172.21.15.19:0/3190056038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d78108de0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.831+0000 7f0d7d526700 1 -- 172.21.15.19:0/3190056038 >> 172.21.15.19:0/3190056038 conn(0x7f0d78104050 msgr2=0x7f0d781064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.831+0000 7f0d7d526700 1 -- 172.21.15.19:0/3190056038 shutdown_connections 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.831+0000 7f0d7d526700 1 -- 172.21.15.19:0/3190056038 wait complete. 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.832+0000 7f0d7d526700 1 Processor -- start 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.832+0000 7f0d7d526700 1 -- start start 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.832+0000 7f0d7d526700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d7819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.832+0000 7f0d7d526700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7810cf50 con 0x7f0d781089d0 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.832+0000 7f0d76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d7819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.832+0000 7f0d76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d7819df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37224/0 (socket says 172.21.15.19:37224) 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.832+0000 7f0d76ffd700 1 -- 172.21.15.19:0/3639620371 learned_addr learned my addr 172.21.15.19:0/3639620371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:33.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d76ffd700 1 -- 172.21.15.19:0/3639620371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7819e460 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d76ffd700 1 --2- 172.21.15.19:0/3639620371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d7819df20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009fd0 tx=0x7f0d7000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d6ffff700 1 -- 172.21.15.19:0/3639620371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d70020020 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7819e6f0 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d6ffff700 1 -- 172.21.15.19:0/3639620371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d70012430 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d6ffff700 1 -- 172.21.15.19:0/3639620371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d70017040 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781a1300 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.833+0000 7f0d6ffff700 1 -- 172.21.15.19:0/3639620371 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d70023020 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.834+0000 7f0d6ffff700 1 -- 172.21.15.19:0/3639620371 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d7000f050 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.834+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d78064cf0 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.837+0000 7f0d6ffff700 1 -- 172.21.15.19:0/3639620371 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0d7002a020 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.909+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f0d7802e510 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.911+0000 7f0d6ffff700 1 -- 172.21.15.19:0/3639620371 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f0d70025d30 con 0x7f0d781089d0 2024-04-25T10:44:33.470 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.912+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 msgr2=0x7f0d7819df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.912+0000 7f0d7d526700 1 --2- 172.21.15.19:0/3639620371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d7819df20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009fd0 tx=0x7f0d7000bd50 comp rx=0 tx=0).stop 2024-04-25T10:44:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.912+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 shutdown_connections 2024-04-25T10:44:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.912+0000 7f0d7d526700 1 --2- 172.21.15.19:0/3639620371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d781089d0 0x7f0d7819df20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.912+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 >> 172.21.15.19:0/3639620371 conn(0x7f0d78104050 msgr2=0x7f0d781944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.912+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 shutdown_connections 2024-04-25T10:44:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:32.912+0000 7f0d7d526700 1 -- 172.21.15.19:0/3639620371 wait complete. 2024-04-25T10:44:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T10:44:33.484 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:44:33.484 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-04-25T10:44:33.485 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T10:44:33.485 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T10:44:33.735 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.x 2024-04-25T10:44:33.735 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T10:44:33.893 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968.target.wants/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@.service. 2024-04-25T10:44:34.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3639620371' entity='client.admin' 2024-04-25T10:44:34.404 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:44:34.404 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T10:44:34.411 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:44:34.412 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T10:44:34.412 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T10:44:34.412 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T10:44:34.420 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:44:34.420 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T10:44:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:44:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T10:44:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T10:44:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T10:44:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-04-25T10:44:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-04-25T10:44:34.557 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:34.482+0000 7f00161e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:44:34.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:34.681+0000 7f00161e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:44:34.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:34.788+0000 7f00161e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:44:35.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:35 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1663686357' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T10:44:35.548 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:35.131+0000 7f00161e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:44:35.548 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:35.223+0000 7f00161e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:44:35.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:35.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9e90b6fe-02f0-11ef-bc93-c7b262605968", 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T10:44:35.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T10:44:35.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T10:44:35.562 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T10:44:35.563 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T10:44:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:44:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T10:44:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T10:44:27.528257+0000", 2024-04-25T10:44:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:44:35.566 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:35.566 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T10:44:35.566 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:44:35.566 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.886+0000 7f3a3a221700 1 Processor -- start 2024-04-25T10:44:35.566 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.887+0000 7f3a3a221700 1 -- start start 2024-04-25T10:44:35.566 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.887+0000 7f3a3a221700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a34108db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.566 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.887+0000 7f3a3a221700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341093a0 con 0x7f3a341089a0 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.887+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a34108db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.887+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a34108db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37254/0 (socket says 172.21.15.19:37254) 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.887+0000 7f3a337fe700 1 -- 172.21.15.19:0/2017508474 learned_addr learned my addr 172.21.15.19:0/2017508474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.888+0000 7f3a337fe700 1 -- 172.21.15.19:0/2017508474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a34109b70 con 0x7f3a341089a0 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.888+0000 7f3a337fe700 1 --2- 172.21.15.19:0/2017508474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a34108db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800bb40 tx=0x7f3a2800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b5576fea2af0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.888+0000 7f3a327fc700 1 -- 172.21.15.19:0/2017508474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a2800dbe0 con 0x7f3a341089a0 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.888+0000 7f3a327fc700 1 -- 172.21.15.19:0/2017508474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a280044c0 con 0x7f3a341089a0 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.888+0000 7f3a327fc700 1 -- 172.21.15.19:0/2017508474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a28010470 con 0x7f3a341089a0 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.889+0000 7f3a3a221700 1 -- 172.21.15.19:0/2017508474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 msgr2=0x7f3a34108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.889+0000 7f3a3a221700 1 --2- 172.21.15.19:0/2017508474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a34108db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800bb40 tx=0x7f3a2800be50 comp rx=0 tx=0).stop 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.889+0000 7f3a3a221700 1 -- 172.21.15.19:0/2017508474 shutdown_connections 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.889+0000 7f3a3a221700 1 --2- 172.21.15.19:0/2017508474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a34108db0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.889+0000 7f3a3a221700 1 -- 172.21.15.19:0/2017508474 >> 172.21.15.19:0/2017508474 conn(0x7f3a34104060 msgr2=0x7f3a341064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.889+0000 7f3a3a221700 1 -- 172.21.15.19:0/2017508474 shutdown_connections 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.889+0000 7f3a3a221700 1 -- 172.21.15.19:0/2017508474 wait complete. 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a3a221700 1 Processor -- start 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a3a221700 1 -- start start 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a3a221700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a341a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a3a221700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3407c170 con 0x7f3a341089a0 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a341a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a341a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37266/0 (socket says 172.21.15.19:37266) 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a337fe700 1 -- 172.21.15.19:0/1663686357 learned_addr learned my addr 172.21.15.19:0/1663686357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.890+0000 7f3a337fe700 1 -- 172.21.15.19:0/1663686357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a341a0630 con 0x7f3a341089a0 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a337fe700 1 --2- 172.21.15.19:0/1663686357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a341a00f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800b340 tx=0x7f3a28003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a30ff9700 1 -- 172.21.15.19:0/1663686357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a28010720 con 0x7f3a341089a0 2024-04-25T10:44:35.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a30ff9700 1 -- 172.21.15.19:0/1663686357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a2801d070 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a341a08c0 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a30ff9700 1 -- 172.21.15.19:0/1663686357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a28019ae0 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a341a34d0 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a30ff9700 1 -- 172.21.15.19:0/1663686357 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a28028060 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.891+0000 7f3a30ff9700 1 -- 172.21.15.19:0/1663686357 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a28021830 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.892+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a34064cf0 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.895+0000 7f3a30ff9700 1 -- 172.21.15.19:0/1663686357 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3a28015070 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.968+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3a341a3740 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.969+0000 7f3a30ff9700 1 -- 172.21.15.19:0/1663686357 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3a28019df0 con 0x7f3a341089a0 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.970+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 msgr2=0x7f3a341a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.970+0000 7f3a3a221700 1 --2- 172.21.15.19:0/1663686357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a341a00f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800b340 tx=0x7f3a28003dc0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.970+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 shutdown_connections 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.970+0000 7f3a3a221700 1 --2- 172.21.15.19:0/1663686357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a341089a0 0x7f3a341a00f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.970+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 >> 172.21.15.19:0/1663686357 conn(0x7f3a34104060 msgr2=0x7f3a34191fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.971+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 shutdown_connections 2024-04-25T10:44:35.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:34.971+0000 7f3a3a221700 1 -- 172.21.15.19:0/1663686357 wait complete. 2024-04-25T10:44:35.570 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-04-25T10:44:36.847 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:36.493+0000 7f00161e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:44:36.847 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:36.668+0000 7f00161e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:44:36.847 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:36.844+0000 7f00161e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:44:37.450 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:37.143+0000 7f00161e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:44:37.452 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:37.238+0000 7f00161e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:44:37.452 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:37.447+0000 7f00161e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:44:37.754 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:37.650+0000 7f00161e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:44:38.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:38 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3699072468' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T10:44:38.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:38.299+0000 7f00161e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:44:38.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:38.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9e90b6fe-02f0-11ef-bc93-c7b262605968", 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T10:44:38.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T10:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T10:44:38.688 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:38.689 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T10:44:27.528257+0000", 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52f1b08700 1 Processor -- start 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52f1b08700 1 -- start start 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52f1b08700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec10ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52f1b08700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec07d670 con 0x7f52ec10a670 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec10ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec10ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37278/0 (socket says 172.21.15.19:37278) 2024-04-25T10:44:38.690 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52eb7fe700 1 -- 172.21.15.19:0/3690760900 learned_addr learned my addr 172.21.15.19:0/3690760900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.968+0000 7f52eb7fe700 1 -- 172.21.15.19:0/3690760900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ec10cfd0 con 0x7f52ec10a670 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.969+0000 7f52eb7fe700 1 --2- 172.21.15.19:0/3690760900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec10ca90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098a0 tx=0x7f52dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c726dd4874944d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.969+0000 7f52ea7fc700 1 -- 172.21.15.19:0/3690760900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52dc004160 con 0x7f52ec10a670 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.969+0000 7f52ea7fc700 1 -- 172.21.15.19:0/3690760900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f52dc004300 con 0x7f52ec10a670 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.969+0000 7f52ea7fc700 1 -- 172.21.15.19:0/3690760900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52dc011af0 con 0x7f52ec10a670 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 -- 172.21.15.19:0/3690760900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 msgr2=0x7f52ec10ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 --2- 172.21.15.19:0/3690760900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec10ca90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098a0 tx=0x7f52dc009bb0 comp rx=0 tx=0).stop 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 -- 172.21.15.19:0/3690760900 shutdown_connections 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 --2- 172.21.15.19:0/3690760900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec10ca90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 -- 172.21.15.19:0/3690760900 >> 172.21.15.19:0/3690760900 conn(0x7f52ec104050 msgr2=0x7f52ec1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 -- 172.21.15.19:0/3690760900 shutdown_connections 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 -- 172.21.15.19:0/3690760900 wait complete. 2024-04-25T10:44:38.691 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.970+0000 7f52f1b08700 1 Processor -- start 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.971+0000 7f52f1b08700 1 -- start start 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.971+0000 7f52f1b08700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.971+0000 7f52f1b08700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10ded0 con 0x7f52ec10a670 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.971+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.971+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37280/0 (socket says 172.21.15.19:37280) 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.971+0000 7f52eb7fe700 1 -- 172.21.15.19:0/3699072468 learned_addr learned my addr 172.21.15.19:0/3699072468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.971+0000 7f52eb7fe700 1 -- 172.21.15.19:0/3699072468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ec19e440 con 0x7f52ec10a670 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52eb7fe700 1 --2- 172.21.15.19:0/3699072468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec19df00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0117b0 tx=0x7f52dc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52e8ff9700 1 -- 172.21.15.19:0/3699072468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52dc0039c0 con 0x7f52ec10a670 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52e8ff9700 1 -- 172.21.15.19:0/3699072468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f52dc003b60 con 0x7f52ec10a670 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ec19e6d0 con 0x7f52ec10a670 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52e8ff9700 1 -- 172.21.15.19:0/3699072468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52dc0196a0 con 0x7f52ec10a670 2024-04-25T10:44:38.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ec1a12e0 con 0x7f52ec10a670 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52e8ff9700 1 -- 172.21.15.19:0/3699072468 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f52dc019cf0 con 0x7f52ec10a670 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.972+0000 7f52e8ff9700 1 -- 172.21.15.19:0/3699072468 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f52dc024730 con 0x7f52ec10a670 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.973+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ec064cf0 con 0x7f52ec10a670 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:37.975+0000 7f52e8ff9700 1 -- 172.21.15.19:0/3699072468 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f52dc01ab00 con 0x7f52ec10a670 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.049+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52ec1a1500 con 0x7f52ec10a670 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.050+0000 7f52e8ff9700 1 -- 172.21.15.19:0/3699072468 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f52dc01a430 con 0x7f52ec10a670 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.051+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 msgr2=0x7f52ec19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.051+0000 7f52f1b08700 1 --2- 172.21.15.19:0/3699072468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec19df00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0117b0 tx=0x7f52dc011b70 comp rx=0 tx=0).stop 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.051+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 shutdown_connections 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.051+0000 7f52f1b08700 1 --2- 172.21.15.19:0/3699072468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ec10a670 0x7f52ec19df00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.051+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 >> 172.21.15.19:0/3699072468 conn(0x7f52ec104050 msgr2=0x7f52ec1049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.051+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 shutdown_connections 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:38.051+0000 7f52f1b08700 1 -- 172.21.15.19:0/3699072468 wait complete. 2024-04-25T10:44:38.693 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-04-25T10:44:39.050 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:38.741+0000 7f00161e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:44:39.050 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:38.840+0000 7f00161e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:44:39.050 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:39.047+0000 7f00161e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:44:39.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:39.590+0000 7f00161e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:44:39.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:39.685+0000 7f00161e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:44:40.293 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:39.936+0000 7f00161e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:44:40.293 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:40.074+0000 7f00161e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:44:40.293 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:40.171+0000 7f00161e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:44:40.293 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:40.290+0000 7f00161e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:44:40.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:40.603+0000 7f00161e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:44:41.519 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:41.241+0000 7f00161e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:44:41.519 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:41.463+0000 7f00161e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:44:41.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: Activating manager daemon x 2024-04-25T10:44:41.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: mgrmap e2: x(active, starting, since 0.0037767s) 2024-04-25T10:44:41.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: Manager daemon x is now available 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' 2024-04-25T10:44:41.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:41 smithi019 ceph-mon[29209]: from='mgr.14100 172.21.15.19:0/2234422257' entity='mgr.x' 2024-04-25T10:44:43.845 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:43 smithi019 ceph-mon[29209]: mgrmap e3: x(active, since 1.00874s) 2024-04-25T10:44:44.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:44 smithi019 ceph-mon[29209]: mgrmap e4: x(active, since 2s) 2024-04-25T10:44:44.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:44 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3184428529' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T10:44:50.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:50.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:44:50.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9e90b6fe-02f0-11ef-bc93-c7b262605968", 2024-04-25T10:44:50.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T10:44:50.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T10:44:50.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T10:44:50.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:50.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T10:44:50.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:50.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T10:44:50.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T10:44:50.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T10:44:50.086 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T10:44:50.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T10:44:27.528257+0000", 2024-04-25T10:44:50.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.539+0000 7f807670b700 1 Processor -- start 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.539+0000 7f807670b700 1 -- start start 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f807670b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f8070108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f807670b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80701093d0 con 0x7f80701089d0 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f806ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f8070108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f806ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f8070108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41712/0 (socket says 172.21.15.19:41712) 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f806ffff700 1 -- 172.21.15.19:0/4022867105 learned_addr learned my addr 172.21.15.19:0/4022867105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:50.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f806ffff700 1 -- 172.21.15.19:0/4022867105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8070109ba0 con 0x7f80701089d0 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f806ffff700 1 --2- 172.21.15.19:0/4022867105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f8070108de0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80600098a0 tx=0x7f8060009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e81dce7259f259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f806effd700 1 -- 172.21.15.19:0/4022867105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8060004160 con 0x7f80701089d0 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.540+0000 7f806effd700 1 -- 172.21.15.19:0/4022867105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8060004300 con 0x7f80701089d0 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f806effd700 1 -- 172.21.15.19:0/4022867105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8060011af0 con 0x7f80701089d0 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f807670b700 1 -- 172.21.15.19:0/4022867105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 msgr2=0x7f8070108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f807670b700 1 --2- 172.21.15.19:0/4022867105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f8070108de0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80600098a0 tx=0x7f8060009bb0 comp rx=0 tx=0).stop 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f807670b700 1 -- 172.21.15.19:0/4022867105 shutdown_connections 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f807670b700 1 --2- 172.21.15.19:0/4022867105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f8070108de0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f807670b700 1 -- 172.21.15.19:0/4022867105 >> 172.21.15.19:0/4022867105 conn(0x7f8070104050 msgr2=0x7f80701064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f807670b700 1 -- 172.21.15.19:0/4022867105 shutdown_connections 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.541+0000 7f807670b700 1 -- 172.21.15.19:0/4022867105 wait complete. 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.542+0000 7f807670b700 1 Processor -- start 2024-04-25T10:44:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.542+0000 7f807670b700 1 -- start start 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.542+0000 7f807670b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f807019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.542+0000 7f807670b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f807010cf50 con 0x7f80701089d0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f807019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f807019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41726/0 (socket says 172.21.15.19:41726) 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806ffff700 1 -- 172.21.15.19:0/3184428529 learned_addr learned my addr 172.21.15.19:0/3184428529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806ffff700 1 -- 172.21.15.19:0/3184428529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807019d350 con 0x7f80701089d0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806ffff700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f807019ce10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8060009fb0 tx=0x7f8060011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806d7fa700 1 -- 172.21.15.19:0/3184428529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80600036a0 con 0x7f80701089d0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806d7fa700 1 -- 172.21.15.19:0/3184428529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8060003cd0 con 0x7f80701089d0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807019d5e0 con 0x7f80701089d0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f806d7fa700 1 -- 172.21.15.19:0/3184428529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f806001a430 con 0x7f80701089d0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.543+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80701a01f0 con 0x7f80701089d0 2024-04-25T10:44:50.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.544+0000 7f806d7fa700 1 -- 172.21.15.19:0/3184428529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f806001a6a0 con 0x7f80701089d0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.544+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8070064cf0 con 0x7f80701089d0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.544+0000 7f806d7fa700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f805803d520 0x7f805803fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.544+0000 7f806d7fa700 1 -- 172.21.15.19:0/3184428529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8060052450 con 0x7f80701089d0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.545+0000 7f806f7fe700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f805803d520 0x7f805803fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.545+0000 7f806f7fe700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f805803d520 0x7f805803fa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8064006fd0 tx=0x7f8064006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.550+0000 7f806d7fa700 1 -- 172.21.15.19:0/3184428529 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f806001aa40 con 0x7f80701089d0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.843+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f80701a0410 con 0x7f80701089d0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.843+0000 7f806d7fa700 1 -- 172.21.15.19:0/3184428529 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8060020240 con 0x7f80701089d0 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f805803d520 msgr2=0x7f805803fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f805803d520 0x7f805803fa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8064006fd0 tx=0x7f8064006e40 comp rx=0 tx=0).stop 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 msgr2=0x7f807019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f807019ce10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8060009fb0 tx=0x7f8060011890 comp rx=0 tx=0).stop 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 shutdown_connections 2024-04-25T10:44:50.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f805803d520 0x7f805803fa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:50.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 --2- 172.21.15.19:0/3184428529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80701089d0 0x7f807019ce10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:50.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 >> 172.21.15.19:0/3184428529 conn(0x7f8070104050 msgr2=0x7f8070104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:50.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 shutdown_connections 2024-04-25T10:44:50.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:43.847+0000 7f807670b700 1 -- 172.21.15.19:0/3184428529 wait complete. 2024-04-25T10:44:50.093 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-04-25T10:44:53.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:53 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/938625971' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T10:44:53.583 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:53.583 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T10:44:53.583 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:44:53.583 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T10:44:53.584 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T10:44:53.584 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T10:44:53.584 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T10:44:53.584 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T10:44:53.584 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:53.584 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T10:44:53.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T10:44:53.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T10:44:53.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T10:44:53.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T10:44:53.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T10:44:53.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T10:44:53.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T10:44:53.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T10:44:53.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T10:44:53.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.771+0000 7f8c34dd6700 1 Processor -- start 2024-04-25T10:44:53.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.771+0000 7f8c34dd6700 1 -- start start 2024-04-25T10:44:53.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.772+0000 7f8c34dd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c30109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:53.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.772+0000 7f8c34dd6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010a4a0 con 0x7f8c30109aa0 2024-04-25T10:44:53.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.772+0000 7f8c2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c30109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:53.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.772+0000 7f8c2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c30109eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56796/0 (socket says 172.21.15.19:56796) 2024-04-25T10:44:53.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.772+0000 7f8c2e59c700 1 -- 172.21.15.19:0/4235890735 learned_addr learned my addr 172.21.15.19:0/4235890735 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:53.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.773+0000 7f8c2e59c700 1 -- 172.21.15.19:0/4235890735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3010ac70 con 0x7f8c30109aa0 2024-04-25T10:44:53.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.773+0000 7f8c2e59c700 1 --2- 172.21.15.19:0/4235890735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c30109eb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c200098a0 tx=0x7f8c20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a4419e7b14695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.773+0000 7f8c2d59a700 1 -- 172.21.15.19:0/4235890735 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c20004160 con 0x7f8c30109aa0 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.773+0000 7f8c2d59a700 1 -- 172.21.15.19:0/4235890735 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c20004300 con 0x7f8c30109aa0 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.773+0000 7f8c2d59a700 1 -- 172.21.15.19:0/4235890735 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c20011af0 con 0x7f8c30109aa0 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.774+0000 7f8c34dd6700 1 -- 172.21.15.19:0/4235890735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 msgr2=0x7f8c30109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.774+0000 7f8c34dd6700 1 --2- 172.21.15.19:0/4235890735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c30109eb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c200098a0 tx=0x7f8c20009bb0 comp rx=0 tx=0).stop 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.774+0000 7f8c34dd6700 1 -- 172.21.15.19:0/4235890735 shutdown_connections 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.774+0000 7f8c34dd6700 1 --2- 172.21.15.19:0/4235890735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c30109eb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.774+0000 7f8c34dd6700 1 -- 172.21.15.19:0/4235890735 >> 172.21.15.19:0/4235890735 conn(0x7f8c301050c0 msgr2=0x7f8c30107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.774+0000 7f8c34dd6700 1 -- 172.21.15.19:0/4235890735 shutdown_connections 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.774+0000 7f8c34dd6700 1 -- 172.21.15.19:0/4235890735 wait complete. 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c34dd6700 1 Processor -- start 2024-04-25T10:44:53.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c34dd6700 1 -- start start 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c34dd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c3007ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c34dd6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010e020 con 0x7f8c30109aa0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c3007ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c3007ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56810/0 (socket says 172.21.15.19:56810) 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c2e59c700 1 -- 172.21.15.19:0/938625971 learned_addr learned my addr 172.21.15.19:0/938625971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.775+0000 7f8c2e59c700 1 -- 172.21.15.19:0/938625971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3007efd0 con 0x7f8c30109aa0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.776+0000 7f8c2e59c700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c3007ea90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c200090e0 tx=0x7f8c20011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.776+0000 7f8c1f7fe700 1 -- 172.21.15.19:0/938625971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c200039e0 con 0x7f8c30109aa0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.776+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3007d220 con 0x7f8c30109aa0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.776+0000 7f8c1f7fe700 1 -- 172.21.15.19:0/938625971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c20003b80 con 0x7f8c30109aa0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.776+0000 7f8c1f7fe700 1 -- 172.21.15.19:0/938625971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c20024690 con 0x7f8c30109aa0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.776+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3007d700 con 0x7f8c30109aa0 2024-04-25T10:44:53.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.777+0000 7f8c1f7fe700 1 -- 172.21.15.19:0/938625971 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f8c20022020 con 0x7f8c30109aa0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.777+0000 7f8c1f7fe700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f8c1803d520 0x7f8c1803fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.777+0000 7f8c1f7fe700 1 -- 172.21.15.19:0/938625971 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c200530c0 con 0x7f8c30109aa0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.777+0000 7f8c2dd9b700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f8c1803d520 0x7f8c1803fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.777+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c30064cf0 con 0x7f8c30109aa0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.778+0000 7f8c2dd9b700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f8c1803d520 0x7f8c1803fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006fd0 tx=0x7f8c24006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.783+0000 7f8c1f7fe700 1 -- 172.21.15.19:0/938625971 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c200172b0 con 0x7f8c30109aa0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.991+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8c3007dca0 con 0x7f8c30109aa0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.992+0000 7f8c1f7fe700 1 -- 172.21.15.19:0/938625971 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f8c2000fdd0 con 0x7f8c30109aa0 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.994+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f8c1803d520 msgr2=0x7f8c1803fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.994+0000 7f8c34dd6700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f8c1803d520 0x7f8c1803fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006fd0 tx=0x7f8c24006e40 comp rx=0 tx=0).stop 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.994+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 msgr2=0x7f8c3007ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:53.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.994+0000 7f8c34dd6700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c3007ea90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c200090e0 tx=0x7f8c20011940 comp rx=0 tx=0).stop 2024-04-25T10:44:53.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.995+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 shutdown_connections 2024-04-25T10:44:53.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.995+0000 7f8c34dd6700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f8c1803d520 0x7f8c1803fa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:53.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.995+0000 7f8c34dd6700 1 --2- 172.21.15.19:0/938625971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c30109aa0 0x7f8c3007ea90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:53.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.995+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 >> 172.21.15.19:0/938625971 conn(0x7f8c301050c0 msgr2=0x7f8c301059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:53.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.995+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 shutdown_connections 2024-04-25T10:44:53.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:52.995+0000 7f8c34dd6700 1 -- 172.21.15.19:0/938625971 wait complete. 2024-04-25T10:44:53.591 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-04-25T10:44:54.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:54 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/36721170' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T10:44:54.551 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:54 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setuser ceph since I am not root 2024-04-25T10:44:54.551 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:54 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setgroup ceph since I am not root 2024-04-25T10:44:54.551 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:54 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:54.549+0000 7fc788747200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:44:54.857 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:54 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:54.747+0000 7fc788747200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:44:54.858 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:54 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:54.855+0000 7fc788747200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:44:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.015+0000 7f91590de700 1 Processor -- start 2024-04-25T10:44:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.016+0000 7f91590de700 1 -- start start 2024-04-25T10:44:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.016+0000 7f91590de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f9154108db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.016+0000 7f91590de700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91541093a0 con 0x7f91541089a0 2024-04-25T10:44:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.016+0000 7f9152d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f9154108db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.016+0000 7f9152d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f9154108db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56826/0 (socket says 172.21.15.19:56826) 2024-04-25T10:44:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.016+0000 7f9152d9d700 1 -- 172.21.15.19:0/2251258114 learned_addr learned my addr 172.21.15.19:0/2251258114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.016+0000 7f9152d9d700 1 -- 172.21.15.19:0/2251258114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9154109b70 con 0x7f91541089a0 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f9152d9d700 1 --2- 172.21.15.19:0/2251258114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f9154108db0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9144009e80 tx=0x7f914400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f3ec5418739c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f9151d9b700 1 -- 172.21.15.19:0/2251258114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9144017070 con 0x7f91541089a0 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f9151d9b700 1 -- 172.21.15.19:0/2251258114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f914400b8c0 con 0x7f91541089a0 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f9151d9b700 1 -- 172.21.15.19:0/2251258114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9144004490 con 0x7f91541089a0 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f91590de700 1 -- 172.21.15.19:0/2251258114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 msgr2=0x7f9154108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f91590de700 1 --2- 172.21.15.19:0/2251258114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f9154108db0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9144009e80 tx=0x7f914400b330 comp rx=0 tx=0).stop 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f91590de700 1 -- 172.21.15.19:0/2251258114 shutdown_connections 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.017+0000 7f91590de700 1 --2- 172.21.15.19:0/2251258114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f9154108db0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.018+0000 7f91590de700 1 -- 172.21.15.19:0/2251258114 >> 172.21.15.19:0/2251258114 conn(0x7f9154104060 msgr2=0x7f91541064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.018+0000 7f91590de700 1 -- 172.21.15.19:0/2251258114 shutdown_connections 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.018+0000 7f91590de700 1 -- 172.21.15.19:0/2251258114 wait complete. 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.018+0000 7f91590de700 1 Processor -- start 2024-04-25T10:44:54.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f91590de700 1 -- start start 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f91590de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f915419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f91590de700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f915410cf20 con 0x7f91541089a0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f9152d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f915419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f9152d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f915419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56834/0 (socket says 172.21.15.19:56834) 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f9152d9d700 1 -- 172.21.15.19:0/36721170 learned_addr learned my addr 172.21.15.19:0/36721170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f9152d9d700 1 -- 172.21.15.19:0/36721170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915419e4b0 con 0x7f91541089a0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.019+0000 7f9152d9d700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f915419df70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9144009fd0 tx=0x7f914400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.020+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9144020020 con 0x7f91541089a0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.020+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9144012430 con 0x7f91541089a0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.020+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915419e740 con 0x7f91541089a0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.020+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9144017040 con 0x7f91541089a0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.020+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a1350 con 0x7f91541089a0 2024-04-25T10:44:54.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.021+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f9144023020 con 0x7f91541089a0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.021+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9154064cf0 con 0x7f91541089a0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.021+0000 7f913bfff700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f913c03d570 0x7f913c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.021+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f914400f050 con 0x7f91541089a0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.021+0000 7f915259c700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f913c03d570 0x7f913c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.022+0000 7f915259c700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f913c03d570 0x7f913c03fa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9148006fd0 tx=0x7f9148006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.026+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f914402a020 con 0x7f91541089a0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.293+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f915402e810 con 0x7f91541089a0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.344+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f914402b260 con 0x7f91541089a0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.346+0000 7f913bfff700 1 -- 172.21.15.19:0/36721170 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f91440125e0 con 0x7f91541089a0 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f913c03d570 msgr2=0x7f913c03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f913c03d570 0x7f913c03fa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9148006fd0 tx=0x7f9148006e40 comp rx=0 tx=0).stop 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 msgr2=0x7f915419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f915419df70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9144009fd0 tx=0x7f914400bd50 comp rx=0 tx=0).stop 2024-04-25T10:44:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 shutdown_connections 2024-04-25T10:44:54.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7f913c03d570 0x7f913c03fa80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:54.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 --2- 172.21.15.19:0/36721170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91541089a0 0x7f915419df70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:54.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 >> 172.21.15.19:0/36721170 conn(0x7f9154104060 msgr2=0x7f9154194520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:54.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 shutdown_connections 2024-04-25T10:44:54.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:54.347+0000 7f91590de700 1 -- 172.21.15.19:0/36721170 wait complete. 2024-04-25T10:44:55.594 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:55 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/36721170' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T10:44:55.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:55 smithi019 ceph-mon[29209]: mgrmap e5: x(active, since 12s) 2024-04-25T10:44:55.595 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:55 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:55.185+0000 7fc788747200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:44:55.595 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:55 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:55.275+0000 7fc788747200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:44:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:44:56 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3093481359' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T10:44:56.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:56 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:56.541+0000 7fc788747200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:44:56.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:44:56.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T10:44:56.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T10:44:56.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T10:44:56.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T10:44:56.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:44:56.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.386+0000 7fd4e7a3f700 1 Processor -- start 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.386+0000 7fd4e7a3f700 1 -- start start 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.387+0000 7fd4e7a3f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e010c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.387+0000 7fd4e7a3f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e007cea0 con 0x7fd4e010a560 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.387+0000 7fd4e57db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e010c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.387+0000 7fd4e57db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e010c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56862/0 (socket says 172.21.15.19:56862) 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.387+0000 7fd4e57db700 1 -- 172.21.15.19:0/728623240 learned_addr learned my addr 172.21.15.19:0/728623240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.387+0000 7fd4e57db700 1 -- 172.21.15.19:0/728623240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e010cec0 con 0x7fd4e010a560 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.387+0000 7fd4e57db700 1 --2- 172.21.15.19:0/728623240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e010c980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0098a0 tx=0x7fd4dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919b8b14af241253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4d7fff700 1 -- 172.21.15.19:0/728623240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4dc004160 con 0x7fd4e010a560 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4d7fff700 1 -- 172.21.15.19:0/728623240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd4dc004300 con 0x7fd4e010a560 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4d7fff700 1 -- 172.21.15.19:0/728623240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4dc011af0 con 0x7fd4e010a560 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/728623240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 msgr2=0x7fd4e010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:56.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4e7a3f700 1 --2- 172.21.15.19:0/728623240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e010c980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0098a0 tx=0x7fd4dc009bb0 comp rx=0 tx=0).stop 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/728623240 shutdown_connections 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4e7a3f700 1 --2- 172.21.15.19:0/728623240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e010c980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.388+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/728623240 >> 172.21.15.19:0/728623240 conn(0x7fd4e0104040 msgr2=0x7fd4e0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.389+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/728623240 shutdown_connections 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.389+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/728623240 wait complete. 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.389+0000 7fd4e7a3f700 1 Processor -- start 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.389+0000 7fd4e7a3f700 1 -- start start 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.389+0000 7fd4e7a3f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.389+0000 7fd4e7a3f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e010de80 con 0x7fd4e010a560 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4e57db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4e57db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56864/0 (socket says 172.21.15.19:56864) 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4e57db700 1 -- 172.21.15.19:0/3093481359 learned_addr learned my addr 172.21.15.19:0/3093481359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:44:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4e57db700 1 -- 172.21.15.19:0/3093481359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e019e460 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4e57db700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e019df20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0117b0 tx=0x7fd4dc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4d67fc700 1 -- 172.21.15.19:0/3093481359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4dc0039c0 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e019e6f0 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4d67fc700 1 -- 172.21.15.19:0/3093481359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd4dc003b60 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4d67fc700 1 -- 172.21.15.19:0/3093481359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4dc0196a0 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.390+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e01a1300 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.391+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e0064cf0 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.392+0000 7fd4d67fc700 1 -- 172.21.15.19:0/3093481359 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fd4dc019960 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.392+0000 7fd4d67fc700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 0x7fd4cc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.392+0000 7fd4d67fc700 1 -- 172.21.15.19:0/3093481359 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4dc052a30 con 0x7fd4e010a560 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.392+0000 7fd4e4fda700 1 -- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 msgr2=0x7fd4cc03fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/370155901 2024-04-25T10:44:56.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.392+0000 7fd4e4fda700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 0x7fd4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.402+0000 7fd4d67fc700 1 -- 172.21.15.19:0/3093481359 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4dc00b3e0 con 0x7fd4e010a560 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.593+0000 7fd4e4fda700 1 -- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 msgr2=0x7fd4cc03fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/370155901 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.593+0000 7fd4e4fda700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 0x7fd4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.673+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd4e01a1980 con 0x7fd4e010a560 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.676+0000 7fd4d67fc700 1 -- 172.21.15.19:0/3093481359 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd4dc0042a0 con 0x7fd4e010a560 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 msgr2=0x7fd4cc03fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 0x7fd4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 msgr2=0x7fd4e019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e019df20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0117b0 tx=0x7fd4dc011b70 comp rx=0 tx=0).stop 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 shutdown_connections 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7fd4cc03d570 0x7fd4cc03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 --2- 172.21.15.19:0/3093481359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e010a560 0x7fd4e019df20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.680+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 >> 172.21.15.19:0/3093481359 conn(0x7fd4e0104040 msgr2=0x7fd4e0105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:56.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.681+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 shutdown_connections 2024-04-25T10:44:56.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:55.681+0000 7fd4e7a3f700 1 -- 172.21.15.19:0/3093481359 wait complete. 2024-04-25T10:44:56.744 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-25T10:44:56.744 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-04-25T10:44:57.041 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:56 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:56.721+0000 7fc788747200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:44:57.041 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:56 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:56.902+0000 7fc788747200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:44:57.311 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:57 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:57.212+0000 7fc788747200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:44:57.311 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:57 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:57.309+0000 7fc788747200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:44:57.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:57 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:57.522+0000 7fc788747200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:44:58.112 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:57 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:57.724+0000 7fc788747200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:44:58.562 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:58 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:58.372+0000 7fc788747200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:44:59.112 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:58 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:58.818+0000 7fc788747200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:44:59.112 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:58 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:58.919+0000 7fc788747200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:44:59.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:59 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:59.131+0000 7fc788747200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:45:00.035 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:59 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:59.682+0000 7fc788747200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:45:00.035 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:44:59 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:44:59.778+0000 7fc788747200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:45:00.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:00.033+0000 7fc788747200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:45:00.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:00.172+0000 7fc788747200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:45:00.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:00.274+0000 7fc788747200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:45:00.715 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:00.396+0000 7fc788747200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:45:01.112 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:00.713+0000 7fc788747200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:45:01.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:01.345+0000 7fc788747200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:45:01.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:01.572+0000 7fc788747200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: Active manager daemon x restarted 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: Activating manager daemon x 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: mgrmap e6: x(active, starting, since 0.00384337s) 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:45:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:45:02.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: Manager daemon x is now available 2024-04-25T10:45:02.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:02.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:02.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:02.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:01 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:02 smithi019 ceph-mon[29209]: Found migration_current of "None". Setting to last migration. 2024-04-25T10:45:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:02 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:45:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:02 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:45:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:02 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:02 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:02 smithi019 ceph-mon[29209]: mgrmap e7: x(active, since 1.00653s) 2024-04-25T10:45:03.122 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:45:03.122 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T10:45:03.122 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T10:45:03.122 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.152+0000 7ff75252d700 1 Processor -- start 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.152+0000 7ff75252d700 1 -- start start 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.152+0000 7ff75252d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c10c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.152+0000 7ff75252d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c07ce40 con 0x7ff74c10a580 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.152+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c10c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.152+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c10c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56870/0 (socket says 172.21.15.19:56870) 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.153+0000 7ff74bfff700 1 -- 172.21.15.19:0/996764799 learned_addr learned my addr 172.21.15.19:0/996764799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.153+0000 7ff74bfff700 1 -- 172.21.15.19:0/996764799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c10cee0 con 0x7ff74c10a580 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.153+0000 7ff74bfff700 1 --2- 172.21.15.19:0/996764799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c10c9a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098a0 tx=0x7ff73c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619c149715a8e504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.153+0000 7ff74affd700 1 -- 172.21.15.19:0/996764799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c004160 con 0x7ff74c10a580 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.153+0000 7ff74affd700 1 -- 172.21.15.19:0/996764799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff73c004300 con 0x7ff74c10a580 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.153+0000 7ff74affd700 1 -- 172.21.15.19:0/996764799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c011af0 con 0x7ff74c10a580 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.154+0000 7ff75252d700 1 -- 172.21.15.19:0/996764799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 msgr2=0x7ff74c10c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.154+0000 7ff75252d700 1 --2- 172.21.15.19:0/996764799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c10c9a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098a0 tx=0x7ff73c009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.154+0000 7ff75252d700 1 -- 172.21.15.19:0/996764799 shutdown_connections 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.154+0000 7ff75252d700 1 --2- 172.21.15.19:0/996764799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c10c9a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.154+0000 7ff75252d700 1 -- 172.21.15.19:0/996764799 >> 172.21.15.19:0/996764799 conn(0x7ff74c104040 msgr2=0x7ff74c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.154+0000 7ff75252d700 1 -- 172.21.15.19:0/996764799 shutdown_connections 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.154+0000 7ff75252d700 1 -- 172.21.15.19:0/996764799 wait complete. 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.155+0000 7ff75252d700 1 Processor -- start 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.155+0000 7ff75252d700 1 -- start start 2024-04-25T10:45:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.155+0000 7ff75252d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.155+0000 7ff75252d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c10de40 con 0x7ff74c10a580 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.155+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.155+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c19dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56874/0 (socket says 172.21.15.19:56874) 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.155+0000 7ff74bfff700 1 -- 172.21.15.19:0/3939014514 learned_addr learned my addr 172.21.15.19:0/3939014514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.156+0000 7ff74bfff700 1 -- 172.21.15.19:0/3939014514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c19e420 con 0x7ff74c10a580 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.156+0000 7ff74bfff700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c19dee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0117b0 tx=0x7ff73c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.156+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c0039c0 con 0x7ff74c10a580 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.156+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff73c003b60 con 0x7ff74c10a580 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.156+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c19e6b0 con 0x7ff74c10a580 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.156+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c0196a0 con 0x7ff74c10a580 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.156+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1a12c0 con 0x7ff74c10a580 2024-04-25T10:45:03.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.157+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7ff73c019910 con 0x7ff74c10a580 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.158+0000 7ff7497fa700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.158+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 --> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff7380401e0 con 0x7ff73803d5c0 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.158+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff73c052900 con 0x7ff74c10a580 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.158+0000 7ff74b7fe700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 msgr2=0x7ff73803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/370155901 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.158+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.359+0000 7ff74b7fe700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 msgr2=0x7ff73803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/370155901 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.359+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.760+0000 7ff74b7fe700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 msgr2=0x7ff73803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/370155901 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:57.760+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:58.561+0000 7ff74b7fe700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 msgr2=0x7ff73803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/370155901 2024-04-25T10:45:03.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:44:58.561+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:45:03.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:00.162+0000 7ff74b7fe700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 msgr2=0x7ff73803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/370155901 2024-04-25T10:45:03.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:00.162+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T10:45:03.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:01.575+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7ff73c019c50 con 0x7ff74c10a580 2024-04-25T10:45:03.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:01.576+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 msgr2=0x7ff73803fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:45:03.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:01.576+0000 7ff7497fa700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:03.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.577+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7ff73c054500 con 0x7ff74c10a580 2024-04-25T10:45:03.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.577+0000 7ff7497fa700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff738040f00 0x7ff738043320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.577+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff7380401e0 con 0x7ff738040f00 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.577+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff738040f00 0x7ff738043320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.578+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff738040f00 0x7ff738043320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff734003a10 tx=0x7ff7340092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.579+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff7380401e0 con 0x7ff738040f00 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff74c064cf0 con 0x7ff738040f00 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff7497fa700 1 -- 172.21.15.19:0/3939014514 <== mgr.14118 v2:172.21.15.19:6800/1932674296 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff74c064cf0 con 0x7ff738040f00 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff738040f00 msgr2=0x7ff738043320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff738040f00 0x7ff738043320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff734003a10 tx=0x7ff7340092b0 comp rx=0 tx=0).stop 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 msgr2=0x7ff74c19dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c19dee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0117b0 tx=0x7ff73c011b70 comp rx=0 tx=0).stop 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 shutdown_connections 2024-04-25T10:45:03.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff738040f00 0x7ff738043320 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:03.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff74c10a580 0x7ff74c19dee0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:03.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 --2- 172.21.15.19:0/3939014514 >> [v2:172.21.15.19:6800/370155901,v1:172.21.15.19:6801/370155901] conn(0x7ff73803d5c0 0x7ff73803fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:03.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 >> 172.21.15.19:0/3939014514 conn(0x7ff74c104040 msgr2=0x7ff74c105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:03.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 shutdown_connections 2024-04-25T10:45:03.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:02.588+0000 7ff75252d700 1 -- 172.21.15.19:0/3939014514 wait complete. 2024-04-25T10:45:03.129 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-04-25T10:45:03.129 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-04-25T10:45:03.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:03 smithi019 ceph-mon[29209]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T10:45:03.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:03 smithi019 ceph-mon[29209]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T10:45:03.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:03 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:02] ENGINE Bus STARTING 2024-04-25T10:45:03.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:03 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:02] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T10:45:03.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:03 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:02] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T10:45:03.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:03 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:02] ENGINE Bus STARTED 2024-04-25T10:45:03.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:03 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:04.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.539+0000 7f78d583a700 1 Processor -- start 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.539+0000 7f78d583a700 1 -- start start 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.539+0000 7f78d583a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d007db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.539+0000 7f78d583a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78d007e0e0 con 0x7f78d007f6e0 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33828/0 (socket says 172.21.15.19:33828) 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78ceffd700 1 -- 172.21.15.19:0/2195864188 learned_addr learned my addr 172.21.15.19:0/2195864188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78ceffd700 1 -- 172.21.15.19:0/2195864188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d007e260 con 0x7f78d007f6e0 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78ceffd700 1 --2- 172.21.15.19:0/2195864188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d007db10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009e80 tx=0x7f78c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a480b497e9a07038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78cdffb700 1 -- 172.21.15.19:0/2195864188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78c0017070 con 0x7f78d007f6e0 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78cdffb700 1 -- 172.21.15.19:0/2195864188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f78c000b8c0 con 0x7f78d007f6e0 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.540+0000 7f78cdffb700 1 -- 172.21.15.19:0/2195864188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78c0004490 con 0x7f78d007f6e0 2024-04-25T10:45:04.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.541+0000 7f78d583a700 1 -- 172.21.15.19:0/2195864188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 msgr2=0x7f78d007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.541+0000 7f78d583a700 1 --2- 172.21.15.19:0/2195864188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d007db10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009e80 tx=0x7f78c000b330 comp rx=0 tx=0).stop 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.541+0000 7f78d583a700 1 -- 172.21.15.19:0/2195864188 shutdown_connections 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.541+0000 7f78d583a700 1 --2- 172.21.15.19:0/2195864188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d007db10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.541+0000 7f78d583a700 1 -- 172.21.15.19:0/2195864188 >> 172.21.15.19:0/2195864188 conn(0x7f78d0105170 msgr2=0x7f78d01075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.541+0000 7f78d583a700 1 -- 172.21.15.19:0/2195864188 shutdown_connections 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.541+0000 7f78d583a700 1 -- 172.21.15.19:0/2195864188 wait complete. 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.542+0000 7f78d583a700 1 Processor -- start 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.542+0000 7f78d583a700 1 -- start start 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.542+0000 7f78d583a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d01a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.542+0000 7f78d583a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78d01100c0 con 0x7f78d007f6e0 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.542+0000 7f78ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d01a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.542+0000 7f78ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d01a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33838/0 (socket says 172.21.15.19:33838) 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.542+0000 7f78ceffd700 1 -- 172.21.15.19:0/3681263275 learned_addr learned my addr 172.21.15.19:0/3681263275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.543+0000 7f78ceffd700 1 -- 172.21.15.19:0/3681263275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d01a0590 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.543+0000 7f78ceffd700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d01a0050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78c0004220 tx=0x7f78c000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.543+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78c0020020 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.543+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f78c0012430 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.543+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d01a0820 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.543+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78c0017040 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.543+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d01a3430 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.544+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d0064cf0 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.544+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7f78c0023020 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.544+0000 7f78d4838700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f78bc03d460 0x7f78bc03f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.544+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f78c000f050 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.544+0000 7f78ce7fc700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f78bc03d460 0x7f78bc03f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.545+0000 7f78ce7fc700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f78bc03d460 0x7f78bc03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f78b8006fd0 tx=0x7f78b8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.550+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78c002a020 con 0x7f78d007f6e0 2024-04-25T10:45:04.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.635+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f78c0024cd0 con 0x7f78d007f6e0 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.778+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f78d0107120 con 0x7f78bc03d460 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.783+0000 7f78d4838700 1 -- 172.21.15.19:0/3681263275 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78d0107120 con 0x7f78bc03d460 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f78bc03d460 msgr2=0x7f78bc03f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f78bc03d460 0x7f78bc03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f78b8006fd0 tx=0x7f78b8006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 msgr2=0x7f78d01a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d01a0050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78c0004220 tx=0x7f78c000bd50 comp rx=0 tx=0).stop 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 shutdown_connections 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f78bc03d460 0x7f78bc03f970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 --2- 172.21.15.19:0/3681263275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d007f6e0 0x7f78d01a0050 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 >> 172.21.15.19:0/3681263275 conn(0x7f78d0105170 msgr2=0x7f78d0106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.785+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 shutdown_connections 2024-04-25T10:45:04.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:03.786+0000 7f78d583a700 1 -- 172.21.15.19:0/3681263275 wait complete. 2024-04-25T10:45:04.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:04 smithi019 ceph-mon[29209]: mgrmap e8: x(active, since 2s) 2024-04-25T10:45:04.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:04 smithi019 ceph-mon[29209]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:04.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:04 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:04.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:04 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:05.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T10:45:05.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.766+0000 7fac0931f700 1 Processor -- start 2024-04-25T10:45:05.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.766+0000 7fac0931f700 1 -- start start 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.766+0000 7fac0931f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac04109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.767+0000 7fac0931f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410a4c0 con 0x7fac04109ac0 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.767+0000 7fac02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac04109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.767+0000 7fac02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac04109ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33852/0 (socket says 172.21.15.19:33852) 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.767+0000 7fac02ffd700 1 -- 172.21.15.19:0/731872749 learned_addr learned my addr 172.21.15.19:0/731872749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.767+0000 7fac02ffd700 1 -- 172.21.15.19:0/731872749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0410ac90 con 0x7fac04109ac0 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.767+0000 7fac02ffd700 1 --2- 172.21.15.19:0/731872749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac04109ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009e80 tx=0x7fabfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe86b4548144708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.767+0000 7fac01ffb700 1 -- 172.21.15.19:0/731872749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabfc017070 con 0x7fac04109ac0 2024-04-25T10:45:05.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac01ffb700 1 -- 172.21.15.19:0/731872749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fabfc00b8c0 con 0x7fac04109ac0 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac01ffb700 1 -- 172.21.15.19:0/731872749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabfc004490 con 0x7fac04109ac0 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac0931f700 1 -- 172.21.15.19:0/731872749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 msgr2=0x7fac04109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac0931f700 1 --2- 172.21.15.19:0/731872749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac04109ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009e80 tx=0x7fabfc00b330 comp rx=0 tx=0).stop 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac0931f700 1 -- 172.21.15.19:0/731872749 shutdown_connections 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac0931f700 1 --2- 172.21.15.19:0/731872749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac04109ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac0931f700 1 -- 172.21.15.19:0/731872749 >> 172.21.15.19:0/731872749 conn(0x7fac041050e0 msgr2=0x7fac041075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac0931f700 1 -- 172.21.15.19:0/731872749 shutdown_connections 2024-04-25T10:45:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.768+0000 7fac0931f700 1 -- 172.21.15.19:0/731872749 wait complete. 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.769+0000 7fac0931f700 1 Processor -- start 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.769+0000 7fac0931f700 1 -- start start 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.769+0000 7fac0931f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac0419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.769+0000 7fac0931f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410e040 con 0x7fac04109ac0 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fac02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac0419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fac02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac0419dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33866/0 (socket says 172.21.15.19:33866) 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fac02ffd700 1 -- 172.21.15.19:0/3871266720 learned_addr learned my addr 172.21.15.19:0/3871266720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fac02ffd700 1 -- 172.21.15.19:0/3871266720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0419e4f0 con 0x7fac04109ac0 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fac02ffd700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac0419dfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009fd0 tx=0x7fabfc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fabfbfff700 1 -- 172.21.15.19:0/3871266720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabfc020020 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fabfbfff700 1 -- 172.21.15.19:0/3871266720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fabfc012430 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fabfbfff700 1 -- 172.21.15.19:0/3871266720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabfc017040 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0419e780 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.770+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac041a1390 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.771+0000 7fabfbfff700 1 -- 172.21.15.19:0/3871266720 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fabfc023020 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.771+0000 7fabfbfff700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fabf003d5c0 0x7fabf003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.771+0000 7fabfbfff700 1 -- 172.21.15.19:0/3871266720 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fabfc00f050 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.772+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac04064cf0 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.772+0000 7fac027fc700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fabf003d5c0 0x7fabf003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.772+0000 7fac027fc700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fabf003d5c0 0x7fabf003fad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabec006fd0 tx=0x7fabec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.777+0000 7fabfbfff700 1 -- 172.21.15.19:0/3871266720 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabfc02a020 con 0x7fac04109ac0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.986+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fac04106120 con 0x7fabf003d5c0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.987+0000 7fabfbfff700 1 -- 172.21.15.19:0/3871266720 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fac04106120 con 0x7fabf003d5c0 2024-04-25T10:45:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fabf003d5c0 msgr2=0x7fabf003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fabf003d5c0 0x7fabf003fad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabec006fd0 tx=0x7fabec006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 msgr2=0x7fac0419dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac0419dfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009fd0 tx=0x7fabfc00bd50 comp rx=0 tx=0).stop 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 shutdown_connections 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fabf003d5c0 0x7fabf003fad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 --2- 172.21.15.19:0/3871266720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac04109ac0 0x7fac0419dfb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 >> 172.21.15.19:0/3871266720 conn(0x7fac041050e0 msgr2=0x7fac04105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 shutdown_connections 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:04.989+0000 7fac0931f700 1 -- 172.21.15.19:0/3871266720 wait complete. 2024-04-25T10:45:05.545 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-04-25T10:45:05.843 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:05 smithi019 ceph-mon[29209]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:06.597 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: Generating public/private rsa key pair. 2024-04-25T10:45:06.597 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: Your identification has been saved in /tmp/tmptnl_zm5a/key. 2024-04-25T10:45:06.597 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: Your public key has been saved in /tmp/tmptnl_zm5a/key.pub. 2024-04-25T10:45:06.597 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: The key fingerprint is: 2024-04-25T10:45:06.597 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: SHA256:dpYHBF90qdUIszIXmcYgcpXN43feSdbZM9Lx6GvXEyk ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:45:06.597 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: The key's randomart image is: 2024-04-25T10:45:06.597 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: +---[RSA 3072]----+ 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | . +o+B*+.+ | 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | o +..O=+..| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | =oo+..*| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | *o.oB=| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | S + .o+o*| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | . o . E.+o| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | ..o| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | o.o| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: | . ..| 2024-04-25T10:45:06.598 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: +----[SHA256]-----+ 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c6acde700 1 Processor -- start 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c6acde700 1 -- start start 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c6acde700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6410ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c6acde700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6407d870 con 0x7f6c6410a5f0 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c68a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6410ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c68a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6410ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33876/0 (socket says 172.21.15.19:33876) 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c68a7a700 1 -- 172.21.15.19:0/1174911110 learned_addr learned my addr 172.21.15.19:0/1174911110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.958+0000 7f6c68a7a700 1 -- 172.21.15.19:0/1174911110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6410cf50 con 0x7f6c6410a5f0 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.959+0000 7f6c68a7a700 1 --2- 172.21.15.19:0/1174911110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6410ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c580098a0 tx=0x7f6c58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e555f40548e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.959+0000 7f6c637fe700 1 -- 172.21.15.19:0/1174911110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c58004160 con 0x7f6c6410a5f0 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.959+0000 7f6c637fe700 1 -- 172.21.15.19:0/1174911110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6c58004300 con 0x7f6c6410a5f0 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.959+0000 7f6c637fe700 1 -- 172.21.15.19:0/1174911110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c58011af0 con 0x7f6c6410a5f0 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.959+0000 7f6c6acde700 1 -- 172.21.15.19:0/1174911110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 msgr2=0x7f6c6410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.959+0000 7f6c6acde700 1 --2- 172.21.15.19:0/1174911110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6410ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c580098a0 tx=0x7f6c58009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 -- 172.21.15.19:0/1174911110 shutdown_connections 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 --2- 172.21.15.19:0/1174911110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6410ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 -- 172.21.15.19:0/1174911110 >> 172.21.15.19:0/1174911110 conn(0x7f6c64103ff0 msgr2=0x7f6c641064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 -- 172.21.15.19:0/1174911110 shutdown_connections 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 -- 172.21.15.19:0/1174911110 wait complete. 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 Processor -- start 2024-04-25T10:45:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 -- start start 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.960+0000 7f6c6acde700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6410deb0 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c68a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c68a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33882/0 (socket says 172.21.15.19:33882) 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c68a7a700 1 -- 172.21.15.19:0/2145776268 learned_addr learned my addr 172.21.15.19:0/2145776268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c68a7a700 1 -- 172.21.15.19:0/2145776268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6419e3e0 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c68a7a700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6419dea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009fb0 tx=0x7f6c58011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c61ffb700 1 -- 172.21.15.19:0/2145776268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c580036a0 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c61ffb700 1 -- 172.21.15.19:0/2145776268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6c5801a430 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6419e670 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.961+0000 7f6c61ffb700 1 -- 172.21.15.19:0/2145776268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c58019550 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.962+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a1280 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.962+0000 7f6c61ffb700 1 -- 172.21.15.19:0/2145776268 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f6c58019800 con 0x7f6c6410a5f0 2024-04-25T10:45:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.962+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c64064cf0 con 0x7f6c6410a5f0 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.963+0000 7f6c61ffb700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f6c5403d570 0x7f6c5403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.963+0000 7f6c61ffb700 1 -- 172.21.15.19:0/2145776268 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6c58052670 con 0x7f6c6410a5f0 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.963+0000 7f6c63fff700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f6c5403d570 0x7f6c5403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.963+0000 7f6c63fff700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f6c5403d570 0x7f6c5403fa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c50006fd0 tx=0x7f6c50006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:05.968+0000 7f6c61ffb700 1 -- 172.21.15.19:0/2145776268 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c58017e90 con 0x7f6c6410a5f0 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.176+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f6c64105060 con 0x7f6c5403d570 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.298+0000 7f6c61ffb700 1 -- 172.21.15.19:0/2145776268 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c64105060 con 0x7f6c5403d570 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f6c5403d570 msgr2=0x7f6c5403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f6c5403d570 0x7f6c5403fa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c50006fd0 tx=0x7f6c50006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 msgr2=0x7f6c6419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6419dea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009fb0 tx=0x7f6c58011940 comp rx=0 tx=0).stop 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 shutdown_connections 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f6c5403d570 0x7f6c5403fa80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 --2- 172.21.15.19:0/2145776268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6410a5f0 0x7f6c6419dea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 >> 172.21.15.19:0/2145776268 conn(0x7f6c64103ff0 msgr2=0x7f6c64104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 shutdown_connections 2024-04-25T10:45:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:06.300+0000 7f6c6acde700 1 -- 172.21.15.19:0/2145776268 wait complete. 2024-04-25T10:45:07.494 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:07 smithi019 ceph-mon[29209]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:07.494 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:07 smithi019 ceph-mon[29209]: Generating ssh key... 2024-04-25T10:45:07.494 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:07 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:07.494 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:07 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:08.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5TcSToHXiFVXiYzVcuvkcLgItB3dWrb1V0rJG4MZIzJGxLOJmSo6LNENmNtMf1+p0lz2uT2Vi79zSz0P3MlGDGw1nZsnbyyKP4OZeMA8Kb1tAuuFj3U4rLgN2UNpYEZIOTLb5zPNaMeK8TQ6XYXuBfLYCISfuGpk7uFZFPZr449diyQusEB/5cFW+hSxhS+VlKb2cP6dhCNLFdcQZrpWcrFgSPadPhI5vsPpi61K73enA60F/C15fuoC6z2D+u/QoV7z3ztJ+CGBw1J0amkrLLHRbxHhUv/mkW+GpzKAArNNet+bfvf/rrJ11Lzz/U1a56jYfP5/B89tFTviLpnUVeuyGcravpvr+ar5CmQduzXd4tWmbJW6VSs6PsrTwa9jZMcc/nHzxdokvWNjjX4jv8dgEdWQqfGi6tlWW6w8c/HNRIKzSDRqM8okqph9KV/5S0PJ+K+0i6o0fGmQ6cT2cH6LBgJ4CGcv+m88+RisBK01jff8nvduwaS1+Rc8NufU= ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:45:08.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.268+0000 7fbdf60e9700 1 Processor -- start 2024-04-25T10:45:08.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.268+0000 7fbdf60e9700 1 -- start start 2024-04-25T10:45:08.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdf60e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf0108d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdf60e9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0109380 con 0x7fbdf0108980 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf0108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf0108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33898/0 (socket says 172.21.15.19:33898) 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdef7fe700 1 -- 172.21.15.19:0/2147125817 learned_addr learned my addr 172.21.15.19:0/2147125817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdef7fe700 1 -- 172.21.15.19:0/2147125817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf0109b50 con 0x7fbdf0108980 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdef7fe700 1 --2- 172.21.15.19:0/2147125817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf0108d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbde00098a0 tx=0x7fbde0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b111450b055558ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.269+0000 7fbdee7fc700 1 -- 172.21.15.19:0/2147125817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde0004160 con 0x7fbdf0108980 2024-04-25T10:45:08.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdee7fc700 1 -- 172.21.15.19:0/2147125817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbde0004300 con 0x7fbdf0108980 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdee7fc700 1 -- 172.21.15.19:0/2147125817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde0011af0 con 0x7fbdf0108980 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2147125817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 msgr2=0x7fbdf0108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdf60e9700 1 --2- 172.21.15.19:0/2147125817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf0108d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbde00098a0 tx=0x7fbde0009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2147125817 shutdown_connections 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdf60e9700 1 --2- 172.21.15.19:0/2147125817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf0108d90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2147125817 >> 172.21.15.19:0/2147125817 conn(0x7fbdf0104040 msgr2=0x7fbdf0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2147125817 shutdown_connections 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.270+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2147125817 wait complete. 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.271+0000 7fbdf60e9700 1 Processor -- start 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.271+0000 7fbdf60e9700 1 -- start start 2024-04-25T10:45:08.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.271+0000 7fbdf60e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdf60e9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf010cf00 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33912/0 (socket says 172.21.15.19:33912) 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdef7fe700 1 -- 172.21.15.19:0/2510589733 learned_addr learned my addr 172.21.15.19:0/2510589733 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdef7fe700 1 -- 172.21.15.19:0/2510589733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf019e4c0 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdef7fe700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf019df80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbde00090e0 tx=0x7fbde0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdecff9700 1 -- 172.21.15.19:0/2510589733 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde00039e0 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdecff9700 1 -- 172.21.15.19:0/2510589733 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbde0003b80 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdecff9700 1 -- 172.21.15.19:0/2510589733 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde0024690 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf019e750 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.272+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf01a1360 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.273+0000 7fbdecff9700 1 -- 172.21.15.19:0/2510589733 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fbde0022020 con 0x7fbdf0108980 2024-04-25T10:45:08.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.273+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf0064cf0 con 0x7fbdf0108980 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.274+0000 7fbdecff9700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fbdd803d5c0 0x7fbdd803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.274+0000 7fbdecff9700 1 -- 172.21.15.19:0/2510589733 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbde0052520 con 0x7fbdf0108980 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.274+0000 7fbdeeffd700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fbdd803d5c0 0x7fbdd803fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.274+0000 7fbdeeffd700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fbdd803d5c0 0x7fbdd803fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006fd0 tx=0x7fbde4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.279+0000 7fbdecff9700 1 -- 172.21.15.19:0/2510589733 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbde00172b0 con 0x7fbdf0108980 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.492+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fbdf002e880 con 0x7fbdd803d5c0 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.493+0000 7fbdecff9700 1 -- 172.21.15.19:0/2510589733 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fbdf002e880 con 0x7fbdd803d5c0 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fbdd803d5c0 msgr2=0x7fbdd803fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fbdd803d5c0 0x7fbdd803fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006fd0 tx=0x7fbde4006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 msgr2=0x7fbdf019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf019df80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbde00090e0 tx=0x7fbde0011940 comp rx=0 tx=0).stop 2024-04-25T10:45:08.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 shutdown_connections 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fbdd803d5c0 0x7fbdd803fad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 --2- 172.21.15.19:0/2510589733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0108980 0x7fbdf019df80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 >> 172.21.15.19:0/2510589733 conn(0x7fbdf0104040 msgr2=0x7fbdf0194550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 shutdown_connections 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:07.495+0000 7fbdf60e9700 1 -- 172.21.15.19:0/2510589733 wait complete. 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T10:45:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-04-25T10:45:08.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:08 smithi019 ceph-mon[29209]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:09.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:09 smithi019 ceph-mon[29209]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:09.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:09 smithi019 ceph-mon[29209]: Deploying cephadm binary to smithi019 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.508+0000 7f273cace700 1 Processor -- start 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.508+0000 7f273cace700 1 -- start start 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.508+0000 7f273cace700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.508+0000 7f273cace700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27381093d0 con 0x7f27381089d0 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.508+0000 7f273659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.508+0000 7f273659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33918/0 (socket says 172.21.15.19:33918) 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.508+0000 7f273659c700 1 -- 172.21.15.19:0/465468929 learned_addr learned my addr 172.21.15.19:0/465468929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.509+0000 7f273659c700 1 -- 172.21.15.19:0/465468929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2738109ba0 con 0x7f27381089d0 2024-04-25T10:45:13.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.509+0000 7f273659c700 1 --2- 172.21.15.19:0/465468929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738108de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27280098a0 tx=0x7f2728009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd73535ec71cae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.509+0000 7f273559a700 1 -- 172.21.15.19:0/465468929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2728004160 con 0x7f27381089d0 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.509+0000 7f273559a700 1 -- 172.21.15.19:0/465468929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2728004300 con 0x7f27381089d0 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.509+0000 7f273559a700 1 -- 172.21.15.19:0/465468929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2728011af0 con 0x7f27381089d0 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.510+0000 7f273cace700 1 -- 172.21.15.19:0/465468929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 msgr2=0x7f2738108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.510+0000 7f273cace700 1 --2- 172.21.15.19:0/465468929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738108de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27280098a0 tx=0x7f2728009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.510+0000 7f273cace700 1 -- 172.21.15.19:0/465468929 shutdown_connections 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.510+0000 7f273cace700 1 --2- 172.21.15.19:0/465468929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738108de0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.510+0000 7f273cace700 1 -- 172.21.15.19:0/465468929 >> 172.21.15.19:0/465468929 conn(0x7f2738104050 msgr2=0x7f27381064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.510+0000 7f273cace700 1 -- 172.21.15.19:0/465468929 shutdown_connections 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.510+0000 7f273cace700 1 -- 172.21.15.19:0/465468929 wait complete. 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273cace700 1 Processor -- start 2024-04-25T10:45:13.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273cace700 1 -- start start 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273cace700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738106260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273cace700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f273810cf50 con 0x7f27381089d0 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33928/0 (socket says 172.21.15.19:33928) 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273659c700 1 -- 172.21.15.19:0/2820111441 learned_addr learned my addr 172.21.15.19:0/2820111441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.511+0000 7f273659c700 1 -- 172.21.15.19:0/2820111441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27381067a0 con 0x7f27381089d0 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.512+0000 7f273659c700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738106260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2728011670 tx=0x7f2728004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.512+0000 7f27277fe700 1 -- 172.21.15.19:0/2820111441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27280036a0 con 0x7f27381089d0 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.512+0000 7f27277fe700 1 -- 172.21.15.19:0/2820111441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2728003cd0 con 0x7f27381089d0 2024-04-25T10:45:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.512+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27381049f0 con 0x7f27381089d0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.512+0000 7f27277fe700 1 -- 172.21.15.19:0/2820111441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27280193c0 con 0x7f27381089d0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.512+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2738104ed0 con 0x7f27381089d0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.513+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2738064cf0 con 0x7f27381089d0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.513+0000 7f27277fe700 1 -- 172.21.15.19:0/2820111441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f2728019630 con 0x7f27381089d0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.513+0000 7f27277fe700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f272003d570 0x7f272003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.513+0000 7f27277fe700 1 -- 172.21.15.19:0/2820111441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2728052510 con 0x7f27381089d0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.513+0000 7f2735d9b700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f272003d570 0x7f272003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.514+0000 7f2735d9b700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f272003d570 0x7f272003fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f272c006fd0 tx=0x7f272c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.518+0000 7f27277fe700 1 -- 172.21.15.19:0/2820111441 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2728017e90 con 0x7f27381089d0 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:08.729+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7f273810cd60 con 0x7f272003d570 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.016+0000 7f27277fe700 1 -- 172.21.15.19:0/2820111441 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f273810cd60 con 0x7f272003d570 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f272003d570 msgr2=0x7f272003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f272003d570 0x7f272003fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f272c006fd0 tx=0x7f272c006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:13.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 msgr2=0x7f2738106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738106260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2728011670 tx=0x7f2728004280 comp rx=0 tx=0).stop 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 shutdown_connections 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f272003d570 0x7f272003fa80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 --2- 172.21.15.19:0/2820111441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27381089d0 0x7f2738106260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 >> 172.21.15.19:0/2820111441 conn(0x7f2738104050 msgr2=0x7f27381944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 shutdown_connections 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:13.019+0000 7f273cace700 1 -- 172.21.15.19:0/2820111441 wait complete. 2024-04-25T10:45:13.673 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-04-25T10:45:14.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:14 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:14.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:14 smithi019 ceph-mon[29209]: Added host smithi019 2024-04-25T10:45:14.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:14 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:15.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:15 smithi019 ceph-mon[29209]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:15.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:15 smithi019 ceph-mon[29209]: Saving service mon spec with placement count:5 2024-04-25T10:45:15.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:15 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:15.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:15 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.137+0000 7f31ca059700 1 Processor -- start 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.137+0000 7f31ca059700 1 -- start start 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.138+0000 7f31ca059700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c410ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.138+0000 7f31ca059700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31c407d850 con 0x7f31c410a630 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.138+0000 7f31c37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c410ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.138+0000 7f31c37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c410ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45530/0 (socket says 172.21.15.19:45530) 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.138+0000 7f31c37fe700 1 -- 172.21.15.19:0/1008915295 learned_addr learned my addr 172.21.15.19:0/1008915295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.138+0000 7f31c37fe700 1 -- 172.21.15.19:0/1008915295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c410cf90 con 0x7f31c410a630 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.138+0000 7f31c37fe700 1 --2- 172.21.15.19:0/1008915295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c410ca50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009e80 tx=0x7f31bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525545b66f53e54e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:15.792 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31c27fc700 1 -- 172.21.15.19:0/1008915295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc017070 con 0x7f31c410a630 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31c27fc700 1 -- 172.21.15.19:0/1008915295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f31bc00b8c0 con 0x7f31c410a630 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31c27fc700 1 -- 172.21.15.19:0/1008915295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc004490 con 0x7f31c410a630 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31ca059700 1 -- 172.21.15.19:0/1008915295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 msgr2=0x7f31c410ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31ca059700 1 --2- 172.21.15.19:0/1008915295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c410ca50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009e80 tx=0x7f31bc00b330 comp rx=0 tx=0).stop 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31ca059700 1 -- 172.21.15.19:0/1008915295 shutdown_connections 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31ca059700 1 --2- 172.21.15.19:0/1008915295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c410ca50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31ca059700 1 -- 172.21.15.19:0/1008915295 >> 172.21.15.19:0/1008915295 conn(0x7f31c4104050 msgr2=0x7f31c41064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31ca059700 1 -- 172.21.15.19:0/1008915295 shutdown_connections 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.139+0000 7f31ca059700 1 -- 172.21.15.19:0/1008915295 wait complete. 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.140+0000 7f31ca059700 1 Processor -- start 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.140+0000 7f31ca059700 1 -- start start 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.140+0000 7f31ca059700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.140+0000 7f31ca059700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31c410def0 con 0x7f31c410a630 2024-04-25T10:45:15.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45538/0 (socket says 172.21.15.19:45538) 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c37fe700 1 -- 172.21.15.19:0/2125834381 learned_addr learned my addr 172.21.15.19:0/2125834381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c37fe700 1 -- 172.21.15.19:0/2125834381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c419e3b0 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c37fe700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c419de70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00b7b0 tx=0x7f31bc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c0ff9700 1 -- 172.21.15.19:0/2125834381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc020020 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c0ff9700 1 -- 172.21.15.19:0/2125834381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f31bc012430 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c419e640 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31c0ff9700 1 -- 172.21.15.19:0/2125834381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc017040 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.141+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c41a1250 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.142+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c4064cf0 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.142+0000 7f31c0ff9700 1 -- 172.21.15.19:0/2125834381 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f31bc023020 con 0x7f31c410a630 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.142+0000 7f31c0ff9700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f31b003d5c0 0x7f31b003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:15.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.142+0000 7f31c0ff9700 1 -- 172.21.15.19:0/2125834381 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f31bc00f050 con 0x7f31c410a630 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.143+0000 7f31c2ffd700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f31b003d5c0 0x7f31b003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.143+0000 7f31c2ffd700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f31b003d5c0 0x7f31b003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31ac006fd0 tx=0x7f31ac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.148+0000 7f31c0ff9700 1 -- 172.21.15.19:0/2125834381 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31bc02a020 con 0x7f31c410a630 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.376+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f31c4104fb0 con 0x7f31b003d5c0 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.382+0000 7f31c0ff9700 1 -- 172.21.15.19:0/2125834381 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f31c4104fb0 con 0x7f31b003d5c0 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f31b003d5c0 msgr2=0x7f31b003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f31b003d5c0 0x7f31b003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31ac006fd0 tx=0x7f31ac006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 msgr2=0x7f31c419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c419de70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00b7b0 tx=0x7f31bc00bd50 comp rx=0 tx=0).stop 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 shutdown_connections 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f31b003d5c0 0x7f31b003fad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 --2- 172.21.15.19:0/2125834381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31c410a630 0x7f31c419de70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 >> 172.21.15.19:0/2125834381 conn(0x7f31c4104050 msgr2=0x7f31c41048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:15.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 shutdown_connections 2024-04-25T10:45:15.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:14.384+0000 7f31ca059700 1 -- 172.21.15.19:0/2125834381 wait complete. 2024-04-25T10:45:15.796 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-04-25T10:45:17.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:16 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:17.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:16 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:17.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.258+0000 7f042000d700 1 Processor -- start 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.258+0000 7f042000d700 1 -- start start 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.258+0000 7f042000d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418108d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.259+0000 7f042000d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0418109380 con 0x7f0418108980 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.259+0000 7f041dda9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.259+0000 7f041dda9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45548/0 (socket says 172.21.15.19:45548) 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.259+0000 7f041dda9700 1 -- 172.21.15.19:0/2047451113 learned_addr learned my addr 172.21.15.19:0/2047451113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.259+0000 7f041dda9700 1 -- 172.21.15.19:0/2047451113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418109b50 con 0x7f0418108980 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.259+0000 7f041dda9700 1 --2- 172.21.15.19:0/2047451113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418108d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04140098a0 tx=0x7f0414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd85f5a9957e40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.259+0000 7f041cda7700 1 -- 172.21.15.19:0/2047451113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0414004160 con 0x7f0418108980 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.260+0000 7f041cda7700 1 -- 172.21.15.19:0/2047451113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0414004300 con 0x7f0418108980 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.260+0000 7f041cda7700 1 -- 172.21.15.19:0/2047451113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0414011a20 con 0x7f0418108980 2024-04-25T10:45:17.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.260+0000 7f042000d700 1 -- 172.21.15.19:0/2047451113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 msgr2=0x7f0418108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.260+0000 7f042000d700 1 --2- 172.21.15.19:0/2047451113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418108d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04140098a0 tx=0x7f0414009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.260+0000 7f042000d700 1 -- 172.21.15.19:0/2047451113 shutdown_connections 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.260+0000 7f042000d700 1 --2- 172.21.15.19:0/2047451113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418108d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.260+0000 7f042000d700 1 -- 172.21.15.19:0/2047451113 >> 172.21.15.19:0/2047451113 conn(0x7f0418104040 msgr2=0x7f0418106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.261+0000 7f042000d700 1 -- 172.21.15.19:0/2047451113 shutdown_connections 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.261+0000 7f042000d700 1 -- 172.21.15.19:0/2047451113 wait complete. 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.261+0000 7f042000d700 1 Processor -- start 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.261+0000 7f042000d700 1 -- start start 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.261+0000 7f042000d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.261+0000 7f042000d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f041810cf00 con 0x7f0418108980 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f041dda9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f041dda9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418107200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45554/0 (socket says 172.21.15.19:45554) 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f041dda9700 1 -- 172.21.15.19:0/2970640878 learned_addr learned my addr 172.21.15.19:0/2970640878 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f041dda9700 1 -- 172.21.15.19:0/2970640878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418107740 con 0x7f0418108980 2024-04-25T10:45:17.769 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f041dda9700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418107200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0414011a20 tx=0x7f0414011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f040effd700 1 -- 172.21.15.19:0/2970640878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04140039c0 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0418105990 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0418105e70 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f040effd700 1 -- 172.21.15.19:0/2970640878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0414003b60 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.262+0000 7f040effd700 1 -- 172.21.15.19:0/2970640878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0414019700 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.263+0000 7f040effd700 1 -- 172.21.15.19:0/2970640878 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f04140199d0 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.263+0000 7f040effd700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f040403d570 0x7f040403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.263+0000 7f040effd700 1 -- 172.21.15.19:0/2970640878 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0414052810 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.264+0000 7f041d5a8700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f040403d570 0x7f040403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.264+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc005330 con 0x7f0418108980 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.264+0000 7f041d5a8700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f040403d570 0x7f040403fa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0408006fd0 tx=0x7f0408006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:17.770 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.274+0000 7f040effd700 1 -- 172.21.15.19:0/2970640878 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0414020e20 con 0x7f0418108980 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.506+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f03fc000bc0 con 0x7f040403d570 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.512+0000 7f040effd700 1 -- 172.21.15.19:0/2970640878 <== mgr.14118 v2:172.21.15.19:6800/1932674296 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f03fc000bc0 con 0x7f040403d570 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f040403d570 msgr2=0x7f040403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f040403d570 0x7f040403fa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0408006fd0 tx=0x7f0408006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 msgr2=0x7f0418107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418107200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0414011a20 tx=0x7f0414011b00 comp rx=0 tx=0).stop 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 shutdown_connections 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f040403d570 0x7f040403fa80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 --2- 172.21.15.19:0/2970640878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418108980 0x7f0418107200 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 >> 172.21.15.19:0/2970640878 conn(0x7f0418104040 msgr2=0x7f04181944a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 shutdown_connections 2024-04-25T10:45:17.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:16.514+0000 7f042000d700 1 -- 172.21.15.19:0/2970640878 wait complete. 2024-04-25T10:45:18.324 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:18 smithi019 ceph-mon[29209]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:18.324 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:18 smithi019 ceph-mon[29209]: Saving service mgr spec with placement count:2 2024-04-25T10:45:18.324 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:18 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:19.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.248+0000 7f1a5e5a2700 1 Processor -- start 2024-04-25T10:45:19.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.248+0000 7f1a5e5a2700 1 -- start start 2024-04-25T10:45:19.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.248+0000 7f1a5e5a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a58109e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:19.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.249+0000 7f1a5e5a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5810a480 con 0x7f1a58109a80 2024-04-25T10:45:19.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.249+0000 7f1a57fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a58109e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:19.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.249+0000 7f1a57fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a58109e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45566/0 (socket says 172.21.15.19:45566) 2024-04-25T10:45:19.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.249+0000 7f1a57fff700 1 -- 172.21.15.19:0/965527166 learned_addr learned my addr 172.21.15.19:0/965527166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.249+0000 7f1a57fff700 1 -- 172.21.15.19:0/965527166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5810ac50 con 0x7f1a58109a80 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.249+0000 7f1a57fff700 1 --2- 172.21.15.19:0/965527166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a58109e90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098a0 tx=0x7f1a48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf789e5eb34eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.250+0000 7f1a56ffd700 1 -- 172.21.15.19:0/965527166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a48004160 con 0x7f1a58109a80 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.250+0000 7f1a56ffd700 1 -- 172.21.15.19:0/965527166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a48004300 con 0x7f1a58109a80 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.250+0000 7f1a56ffd700 1 -- 172.21.15.19:0/965527166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a48011a20 con 0x7f1a58109a80 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.251+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/965527166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 msgr2=0x7f1a58109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.251+0000 7f1a5e5a2700 1 --2- 172.21.15.19:0/965527166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a58109e90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098a0 tx=0x7f1a48009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.251+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/965527166 shutdown_connections 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.251+0000 7f1a5e5a2700 1 --2- 172.21.15.19:0/965527166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a58109e90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.251+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/965527166 >> 172.21.15.19:0/965527166 conn(0x7f1a58105160 msgr2=0x7f1a581075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.251+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/965527166 shutdown_connections 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.251+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/965527166 wait complete. 2024-04-25T10:45:19.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.252+0000 7f1a5e5a2700 1 Processor -- start 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.252+0000 7f1a5e5a2700 1 -- start start 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.252+0000 7f1a5e5a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a5819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.252+0000 7f1a5e5a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5810e000 con 0x7f1a58109a80 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.252+0000 7f1a57fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a5819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.252+0000 7f1a57fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a5819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45568/0 (socket says 172.21.15.19:45568) 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.252+0000 7f1a57fff700 1 -- 172.21.15.19:0/1760626061 learned_addr learned my addr 172.21.15.19:0/1760626061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.253+0000 7f1a57fff700 1 -- 172.21.15.19:0/1760626061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5819e420 con 0x7f1a58109a80 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.253+0000 7f1a57fff700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a5819dee0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a48003e70 tx=0x7f1a48011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.253+0000 7f1a557fa700 1 -- 172.21.15.19:0/1760626061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a48003af0 con 0x7f1a58109a80 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.253+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5819e6b0 con 0x7f1a58109a80 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.253+0000 7f1a557fa700 1 -- 172.21.15.19:0/1760626061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4801a8e0 con 0x7f1a58109a80 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.253+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a581a12c0 con 0x7f1a58109a80 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.253+0000 7f1a557fa700 1 -- 172.21.15.19:0/1760626061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a48024870 con 0x7f1a58109a80 2024-04-25T10:45:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.254+0000 7f1a557fa700 1 -- 172.21.15.19:0/1760626061 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f1a48022020 con 0x7f1a58109a80 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.254+0000 7f1a557fa700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f1a4003d520 0x7f1a4003fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.254+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a58064cf0 con 0x7f1a58109a80 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.254+0000 7f1a557fa700 1 -- 172.21.15.19:0/1760626061 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a48053610 con 0x7f1a58109a80 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.255+0000 7f1a577fe700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f1a4003d520 0x7f1a4003fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.255+0000 7f1a577fe700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f1a4003d520 0x7f1a4003fa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c006fd0 tx=0x7f1a4c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.260+0000 7f1a557fa700 1 -- 172.21.15.19:0/1760626061 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a4800e260 con 0x7f1a58109a80 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.482+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1a581a1890 con 0x7f1a58109a80 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.485+0000 7f1a557fa700 1 -- 172.21.15.19:0/1760626061 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f1a4801ad90 con 0x7f1a58109a80 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f1a4003d520 msgr2=0x7f1a4003fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f1a4003d520 0x7f1a4003fa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c006fd0 tx=0x7f1a4c006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 msgr2=0x7f1a5819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a5819dee0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a48003e70 tx=0x7f1a48011aa0 comp rx=0 tx=0).stop 2024-04-25T10:45:19.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 shutdown_connections 2024-04-25T10:45:19.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f1a4003d520 0x7f1a4003fa30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:19.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 --2- 172.21.15.19:0/1760626061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a58109a80 0x7f1a5819dee0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:19.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 >> 172.21.15.19:0/1760626061 conn(0x7f1a58105160 msgr2=0x7f1a58106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:19.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 shutdown_connections 2024-04-25T10:45:19.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:18.488+0000 7f1a5e5a2700 1 -- 172.21.15.19:0/1760626061 wait complete. 2024-04-25T10:45:19.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:19 smithi019 ceph-mon[29209]: from='mgr.14118 172.21.15.19:0/654518102' entity='mgr.x' 2024-04-25T10:45:19.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:19 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1760626061' entity='client.admin' 2024-04-25T10:45:20.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.534+0000 7ff663ac2700 1 Processor -- start 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.534+0000 7ff663ac2700 1 -- start start 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff663ac2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c10c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff663ac2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c07cea0 con 0x7ff65c10a560 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66185e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c10c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66185e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c10c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45582/0 (socket says 172.21.15.19:45582) 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66185e700 1 -- 172.21.15.19:0/625972634 learned_addr learned my addr 172.21.15.19:0/625972634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66185e700 1 -- 172.21.15.19:0/625972634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c10cec0 con 0x7ff65c10a560 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66185e700 1 --2- 172.21.15.19:0/625972634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c10c980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009e80 tx=0x7ff64c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f30ebab6ceef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66085c700 1 -- 172.21.15.19:0/625972634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff64c017070 con 0x7ff65c10a560 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66085c700 1 -- 172.21.15.19:0/625972634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff64c00b8c0 con 0x7ff65c10a560 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.535+0000 7ff66085c700 1 -- 172.21.15.19:0/625972634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff64c004490 con 0x7ff65c10a560 2024-04-25T10:45:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.536+0000 7ff663ac2700 1 -- 172.21.15.19:0/625972634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 msgr2=0x7ff65c10c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.536+0000 7ff663ac2700 1 --2- 172.21.15.19:0/625972634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c10c980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009e80 tx=0x7ff64c00b330 comp rx=0 tx=0).stop 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.536+0000 7ff663ac2700 1 -- 172.21.15.19:0/625972634 shutdown_connections 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.536+0000 7ff663ac2700 1 --2- 172.21.15.19:0/625972634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c10c980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.536+0000 7ff663ac2700 1 -- 172.21.15.19:0/625972634 >> 172.21.15.19:0/625972634 conn(0x7ff65c104040 msgr2=0x7ff65c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.536+0000 7ff663ac2700 1 -- 172.21.15.19:0/625972634 shutdown_connections 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.536+0000 7ff663ac2700 1 -- 172.21.15.19:0/625972634 wait complete. 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.537+0000 7ff663ac2700 1 Processor -- start 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.537+0000 7ff663ac2700 1 -- start start 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.537+0000 7ff663ac2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.537+0000 7ff663ac2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10dec0 con 0x7ff65c10a560 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.537+0000 7ff66185e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.537+0000 7ff66185e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45598/0 (socket says 172.21.15.19:45598) 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.537+0000 7ff66185e700 1 -- 172.21.15.19:0/3683354269 learned_addr learned my addr 172.21.15.19:0/3683354269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:20.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.538+0000 7ff66185e700 1 -- 172.21.15.19:0/3683354269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c19e440 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.538+0000 7ff66185e700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c19df00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0072a0 tx=0x7ff64c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.538+0000 7ff65affd700 1 -- 172.21.15.19:0/3683354269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff64c020020 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.538+0000 7ff65affd700 1 -- 172.21.15.19:0/3683354269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff64c012430 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.538+0000 7ff65affd700 1 -- 172.21.15.19:0/3683354269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff64c017040 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.538+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c19e6d0 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.538+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c1a12e0 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.539+0000 7ff65affd700 1 -- 172.21.15.19:0/3683354269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7ff64c023020 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.539+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c064cf0 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.539+0000 7ff65affd700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff64403d570 0x7ff64403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.539+0000 7ff65affd700 1 -- 172.21.15.19:0/3683354269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff64c00f050 con 0x7ff65c10a560 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.540+0000 7ff66105d700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff64403d570 0x7ff64403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.540+0000 7ff66105d700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff64403d570 0x7ff64403fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff650006fd0 tx=0x7ff650006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.544+0000 7ff65affd700 1 -- 172.21.15.19:0/3683354269 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff64c02a020 con 0x7ff65c10a560 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.751+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff65c1a14b0 con 0x7ff65c10a560 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.755+0000 7ff65affd700 1 -- 172.21.15.19:0/3683354269 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff64c003a20 con 0x7ff65c10a560 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff64403d570 msgr2=0x7ff64403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff64403d570 0x7ff64403fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff650006fd0 tx=0x7ff650006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 msgr2=0x7ff65c19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c19df00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0072a0 tx=0x7ff64c00bd50 comp rx=0 tx=0).stop 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 shutdown_connections 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7ff64403d570 0x7ff64403fa80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 --2- 172.21.15.19:0/3683354269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff65c10a560 0x7ff65c19df00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 >> 172.21.15.19:0/3683354269 conn(0x7ff65c104040 msgr2=0x7ff65c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 shutdown_connections 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:19.758+0000 7ff663ac2700 1 -- 172.21.15.19:0/3683354269 wait complete. 2024-04-25T10:45:20.294 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-04-25T10:45:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:20 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3683354269' entity='client.admin' 2024-04-25T10:45:22.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:22 smithi019 ceph-mon[29209]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:45:22.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:22 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3880981806' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T10:45:22.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setuser ceph since I am not root 2024-04-25T10:45:22.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setgroup ceph since I am not root 2024-04-25T10:45:22.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:22.813+0000 7f6a68639200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:45:23.234 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:23.009+0000 7f6a68639200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:45:23.234 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:23.117+0000 7f6a68639200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.617+0000 7fcfa5ac8700 1 Processor -- start 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.617+0000 7fcfa5ac8700 1 -- start start 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.617+0000 7fcfa5ac8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0108d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.617+0000 7fcfa5ac8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0109380 con 0x7fcfa0108980 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45614/0 (socket says 172.21.15.19:45614) 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9f7fe700 1 -- 172.21.15.19:0/1030073493 learned_addr learned my addr 172.21.15.19:0/1030073493 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9f7fe700 1 -- 172.21.15.19:0/1030073493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa0109b50 con 0x7fcfa0108980 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9f7fe700 1 --2- 172.21.15.19:0/1030073493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0108d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009e80 tx=0x7fcf9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3da3aee8940d04c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9e7fc700 1 -- 172.21.15.19:0/1030073493 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf98017070 con 0x7fcfa0108980 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9e7fc700 1 -- 172.21.15.19:0/1030073493 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf9800b8c0 con 0x7fcfa0108980 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.618+0000 7fcf9e7fc700 1 -- 172.21.15.19:0/1030073493 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf98004490 con 0x7fcfa0108980 2024-04-25T10:45:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.619+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/1030073493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 msgr2=0x7fcfa0108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.619+0000 7fcfa5ac8700 1 --2- 172.21.15.19:0/1030073493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0108d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009e80 tx=0x7fcf9800b330 comp rx=0 tx=0).stop 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.619+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/1030073493 shutdown_connections 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.619+0000 7fcfa5ac8700 1 --2- 172.21.15.19:0/1030073493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0108d90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.619+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/1030073493 >> 172.21.15.19:0/1030073493 conn(0x7fcfa0103fc0 msgr2=0x7fcfa0106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.619+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/1030073493 shutdown_connections 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.619+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/1030073493 wait complete. 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.620+0000 7fcfa5ac8700 1 Processor -- start 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.620+0000 7fcfa5ac8700 1 -- start start 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.620+0000 7fcfa5ac8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0106120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.620+0000 7fcfa5ac8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa010cf00 con 0x7fcfa0108980 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0106120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0106120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45628/0 (socket says 172.21.15.19:45628) 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9f7fe700 1 -- 172.21.15.19:0/3880981806 learned_addr learned my addr 172.21.15.19:0/3880981806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:23.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9f7fe700 1 -- 172.21.15.19:0/3880981806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa0106660 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9f7fe700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0106120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009e50 tx=0x7fcf9800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf98020020 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98012430 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa01048b0 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf98017040 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.621+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa0104df0 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.622+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fcf98023020 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.622+0000 7fcf9cff9700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fcf8c03d520 0x7fcf8c03fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.623+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf9800f050 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.623+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa0064cf0 con 0x7fcfa0108980 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.623+0000 7fcf9effd700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fcf8c03d520 0x7fcf8c03fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:23.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.623+0000 7fcf9effd700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fcf8c03d520 0x7fcf8c03fa30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf88006fd0 tx=0x7fcf88006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.628+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf9802a020 con 0x7fcfa0108980 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:21.897+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcfa0196fb0 con 0x7fcfa0108980 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.586+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcf9802dd60 con 0x7fcfa0108980 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.589+0000 7fcf9cff9700 1 -- 172.21.15.19:0/3880981806 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fcf98024cb0 con 0x7fcfa0108980 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fcf8c03d520 msgr2=0x7fcf8c03fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fcf8c03d520 0x7fcf8c03fa30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf88006fd0 tx=0x7fcf88006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 msgr2=0x7fcfa0106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0106120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009e50 tx=0x7fcf9800bd50 comp rx=0 tx=0).stop 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 shutdown_connections 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fcf8c03d520 0x7fcf8c03fa30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 --2- 172.21.15.19:0/3880981806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa0108980 0x7fcfa0106120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 >> 172.21.15.19:0/3880981806 conn(0x7fcfa0103fc0 msgr2=0x7fcfa01944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 shutdown_connections 2024-04-25T10:45:23.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:22.590+0000 7fcfa5ac8700 1 -- 172.21.15.19:0/3880981806 wait complete. 2024-04-25T10:45:23.490 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:23.453+0000 7f6a68639200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:45:23.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:23 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3880981806' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T10:45:23.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:23 smithi019 ceph-mon[29209]: mgrmap e9: x(active, since 21s) 2024-04-25T10:45:23.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:23.548+0000 7f6a68639200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:45:24.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.682+0000 7f7710db7700 1 Processor -- start 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.682+0000 7f7710db7700 1 -- start start 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.682+0000 7f7710db7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c10c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.682+0000 7f7710db7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f770c07d8d0 con 0x7f770c10a590 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c10c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:24.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c10c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58126/0 (socket says 172.21.15.19:58126) 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770a59c700 1 -- 172.21.15.19:0/2830573835 learned_addr learned my addr 172.21.15.19:0/2830573835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770a59c700 1 -- 172.21.15.19:0/2830573835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c10cef0 con 0x7f770c10a590 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770a59c700 1 --2- 172.21.15.19:0/2830573835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c10c9b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098a0 tx=0x7f76fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aec4d0b73bbe790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770959a700 1 -- 172.21.15.19:0/2830573835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc004160 con 0x7f770c10a590 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770959a700 1 -- 172.21.15.19:0/2830573835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76fc004300 con 0x7f770c10a590 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.683+0000 7f770959a700 1 -- 172.21.15.19:0/2830573835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc011a20 con 0x7f770c10a590 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.684+0000 7f7710db7700 1 -- 172.21.15.19:0/2830573835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 msgr2=0x7f770c10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.684+0000 7f7710db7700 1 --2- 172.21.15.19:0/2830573835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c10c9b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098a0 tx=0x7f76fc009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.684+0000 7f7710db7700 1 -- 172.21.15.19:0/2830573835 shutdown_connections 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.684+0000 7f7710db7700 1 --2- 172.21.15.19:0/2830573835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c10c9b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.684+0000 7f7710db7700 1 -- 172.21.15.19:0/2830573835 >> 172.21.15.19:0/2830573835 conn(0x7f770c104050 msgr2=0x7f770c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.684+0000 7f7710db7700 1 -- 172.21.15.19:0/2830573835 shutdown_connections 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.684+0000 7f7710db7700 1 -- 172.21.15.19:0/2830573835 wait complete. 2024-04-25T10:45:24.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.685+0000 7f7710db7700 1 Processor -- start 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.685+0000 7f7710db7700 1 -- start start 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.685+0000 7f7710db7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.685+0000 7f7710db7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f770c10def0 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f770a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f770a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58132/0 (socket says 172.21.15.19:58132) 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f770a59c700 1 -- 172.21.15.19:0/650190556 learned_addr learned my addr 172.21.15.19:0/650190556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f770a59c700 1 -- 172.21.15.19:0/650190556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c19e3f0 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f770a59c700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c19deb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009520 tx=0x7f76fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f76fb7fe700 1 -- 172.21.15.19:0/650190556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc0039e0 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f76fb7fe700 1 -- 172.21.15.19:0/650190556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76fc01b8f0 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c19e680 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f76fb7fe700 1 -- 172.21.15.19:0/650190556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc0197d0 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.686+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c1a1290 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.687+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c064cf0 con 0x7f770c10a590 2024-04-25T10:45:24.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.687+0000 7f76fb7fe700 1 -- 172.21.15.19:0/650190556 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f76fc019a90 con 0x7f770c10a590 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.688+0000 7f76fb7fe700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 0x7f76f403fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.688+0000 7f7709d9b700 1 -- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 msgr2=0x7f76f403fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1932674296 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.688+0000 7f76fb7fe700 1 -- 172.21.15.19:0/650190556 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f76fc052e50 con 0x7f770c10a590 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.688+0000 7f7709d9b700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 0x7f76f403fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.693+0000 7f76fb7fe700 1 -- 172.21.15.19:0/650190556 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76fc01f030 con 0x7f770c10a590 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.889+0000 7f7709d9b700 1 -- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 msgr2=0x7f76f403fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1932674296 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:23.889+0000 7f7709d9b700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 0x7f76f403fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.003+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f770c1a1460 con 0x7f770c10a590 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.004+0000 7f76fb7fe700 1 -- 172.21.15.19:0/650190556 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f76fc02a350 con 0x7f770c10a590 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.008+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 msgr2=0x7f76f403fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.008+0000 7f7710db7700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 0x7f76f403fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.008+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 msgr2=0x7f770c19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:24.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.008+0000 7f7710db7700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c19deb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009520 tx=0x7f76fc011aa0 comp rx=0 tx=0).stop 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.009+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 shutdown_connections 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.009+0000 7f7710db7700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7f76f403d5c0 0x7f76f403fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.009+0000 7f7710db7700 1 --2- 172.21.15.19:0/650190556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f770c10a590 0x7f770c19deb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.009+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 >> 172.21.15.19:0/650190556 conn(0x7f770c104050 msgr2=0x7f770c104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.009+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 shutdown_connections 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:24.009+0000 7f7710db7700 1 -- 172.21.15.19:0/650190556 wait complete. 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-25T10:45:24.595 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-04-25T10:45:24.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/650190556' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T10:45:24.834 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:24.820+0000 7f6a68639200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:45:25.111 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:25.005+0000 7f6a68639200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:45:25.498 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:25.189+0000 7f6a68639200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:45:25.498 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:25.496+0000 7f6a68639200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:45:25.808 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:25.594+0000 7f6a68639200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:45:25.808 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:25.806+0000 7f6a68639200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:45:26.111 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:26.010+0000 7f6a68639200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:45:26.861 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:26.642+0000 7f6a68639200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:45:27.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:27.090+0000 7f6a68639200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:45:27.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:27.192+0000 7f6a68639200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:45:27.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:27.405+0000 7f6a68639200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:45:28.314 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:27.954+0000 7f6a68639200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:45:28.314 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:28.052+0000 7f6a68639200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:45:28.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:28.312+0000 7f6a68639200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:45:28.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:28.453+0000 7f6a68639200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:45:28.612 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:28.553+0000 7f6a68639200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:45:29.000 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:28.675+0000 7f6a68639200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:45:29.000 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:28.998+0000 7f6a68639200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:45:29.955 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:29.667+0000 7f6a68639200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:45:29.955 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:45:29.900+0000 7f6a68639200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: Active manager daemon x restarted 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: Activating manager daemon x 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: mgrmap e10: x(active, starting, since 0.00451488s) 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:45:30.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: Manager daemon x is now available 2024-04-25T10:45:30.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:31.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:45:31.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:45:31.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:30 smithi019 ceph-mon[29209]: mgrmap e11: x(active, since 1.00716s) 2024-04-25T10:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T10:45:31.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.058+0000 7fd891bbf700 1 Processor -- start 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.058+0000 7fd891bbf700 1 -- start start 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.059+0000 7fd891bbf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c07d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.059+0000 7fd891bbf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c07d7a0 con 0x7fd88c07ee30 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.059+0000 7fd88b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c07d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.059+0000 7fd88b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c07d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58144/0 (socket says 172.21.15.19:58144) 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.059+0000 7fd88b7fe700 1 -- 172.21.15.19:0/3135812820 learned_addr learned my addr 172.21.15.19:0/3135812820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.059+0000 7fd88b7fe700 1 -- 172.21.15.19:0/3135812820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c07d920 con 0x7fd88c07ee30 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.059+0000 7fd88b7fe700 1 --2- 172.21.15.19:0/3135812820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c07d260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8740098a0 tx=0x7fd874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6198ba9f5ddc93e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.060+0000 7fd88a7fc700 1 -- 172.21.15.19:0/3135812820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd874004160 con 0x7fd88c07ee30 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.060+0000 7fd88a7fc700 1 -- 172.21.15.19:0/3135812820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd874004300 con 0x7fd88c07ee30 2024-04-25T10:45:31.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.060+0000 7fd88a7fc700 1 -- 172.21.15.19:0/3135812820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd874011a20 con 0x7fd88c07ee30 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.060+0000 7fd891bbf700 1 -- 172.21.15.19:0/3135812820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 msgr2=0x7fd88c07d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.060+0000 7fd891bbf700 1 --2- 172.21.15.19:0/3135812820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c07d260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8740098a0 tx=0x7fd874009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.061+0000 7fd891bbf700 1 -- 172.21.15.19:0/3135812820 shutdown_connections 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.061+0000 7fd891bbf700 1 --2- 172.21.15.19:0/3135812820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c07d260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.061+0000 7fd891bbf700 1 -- 172.21.15.19:0/3135812820 >> 172.21.15.19:0/3135812820 conn(0x7fd88c1050c0 msgr2=0x7fd88c107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.061+0000 7fd891bbf700 1 -- 172.21.15.19:0/3135812820 shutdown_connections 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.061+0000 7fd891bbf700 1 -- 172.21.15.19:0/3135812820 wait complete. 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.061+0000 7fd891bbf700 1 Processor -- start 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd891bbf700 1 -- start start 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd891bbf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd891bbf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1100a0 con 0x7fd88c07ee30 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd88b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd88b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c1a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58156/0 (socket says 172.21.15.19:58156) 2024-04-25T10:45:31.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd88b7fe700 1 -- 172.21.15.19:0/2002742757 learned_addr learned my addr 172.21.15.19:0/2002742757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd88b7fe700 1 -- 172.21.15.19:0/2002742757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c1a05a0 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.062+0000 7fd88b7fe700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c1a0060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd874011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.063+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd874003aa0 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.063+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c1a0830 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.063+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd87401b8f0 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.063+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd874023890 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.063+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1a3440 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.064+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fd87402c3f0 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.064+0000 7fd888ff9700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.064+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 --> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd8780401e0 con 0x7fd87803d5c0 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.064+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd874052ca0 con 0x7fd88c07ee30 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.064+0000 7fd88affd700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 msgr2=0x7fd87803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1932674296 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.064+0000 7fd88affd700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:45:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.265+0000 7fd88affd700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 msgr2=0x7fd87803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1932674296 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.265+0000 7fd88affd700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.666+0000 7fd88affd700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 msgr2=0x7fd87803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1932674296 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:25.666+0000 7fd88affd700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:26.467+0000 7fd88affd700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 msgr2=0x7fd87803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1932674296 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:26.467+0000 7fd88affd700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:28.068+0000 7fd88affd700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 msgr2=0x7fd87803fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1932674296 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:28.068+0000 7fd88affd700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:29.904+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7fd87402cb00 con 0x7fd88c07ee30 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:29.904+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 msgr2=0x7fd87803fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:29.904+0000 7fd888ff9700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.905+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7fd87402bcc0 con 0x7fd88c07ee30 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.906+0000 7fd888ff9700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd874052830 0x7fd878043000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:31.874 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.906+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd874051220 con 0x7fd874052830 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.907+0000 7fd88affd700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd874052830 0x7fd878043000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.907+0000 7fd88affd700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd874052830 0x7fd878043000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd87c003a10 tx=0x7fd87c0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.908+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd874051220 con 0x7fd874052830 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.916+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd88c064cf0 con 0x7fd874052830 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd888ff9700 1 -- 172.21.15.19:0/2002742757 <== mgr.14150 v2:172.21.15.19:6800/1708221429 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd88c064cf0 con 0x7fd874052830 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd874052830 msgr2=0x7fd878043000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd874052830 0x7fd878043000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd87c003a10 tx=0x7fd87c0092b0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 msgr2=0x7fd88c1a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c1a0060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd874011aa0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 shutdown_connections 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd874052830 0x7fd878043000 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88c07ee30 0x7fd88c1a0060 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 --2- 172.21.15.19:0/2002742757 >> [v2:172.21.15.19:6800/1932674296,v1:172.21.15.19:6801/1932674296] conn(0x7fd87803d5c0 0x7fd87803fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 >> 172.21.15.19:0/2002742757 conn(0x7fd88c1050c0 msgr2=0x7fd88c105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.917+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 shutdown_connections 2024-04-25T10:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:30.918+0000 7fd891bbf700 1 -- 172.21.15.19:0/2002742757 wait complete. 2024-04-25T10:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-04-25T10:45:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-04-25T10:45:32.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:32 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:30] ENGINE Bus STARTING 2024-04-25T10:45:32.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:32 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:31] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T10:45:32.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:32 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:31] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T10:45:32.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:32 smithi019 ceph-mon[29209]: [25/Apr/2024:10:45:31] ENGINE Bus STARTED 2024-04-25T10:45:32.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:33.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T10:45:33.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.320+0000 7fa7dc7d6700 1 Processor -- start 2024-04-25T10:45:33.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.320+0000 7fa7dc7d6700 1 -- start start 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.321+0000 7fa7dc7d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d4109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.321+0000 7fa7dc7d6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d410a4f0 con 0x7fa7d4109af0 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.321+0000 7fa7da572700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d4109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.321+0000 7fa7da572700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d4109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58236/0 (socket says 172.21.15.19:58236) 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.321+0000 7fa7da572700 1 -- 172.21.15.19:0/417672903 learned_addr learned my addr 172.21.15.19:0/417672903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.321+0000 7fa7da572700 1 -- 172.21.15.19:0/417672903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d410acc0 con 0x7fa7d4109af0 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.321+0000 7fa7da572700 1 --2- 172.21.15.19:0/417672903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d4109f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80098a0 tx=0x7fa7c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d1a6fd7f11d02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.322+0000 7fa7d9570700 1 -- 172.21.15.19:0/417672903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7c8004160 con 0x7fa7d4109af0 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.322+0000 7fa7d9570700 1 -- 172.21.15.19:0/417672903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c8004300 con 0x7fa7d4109af0 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.322+0000 7fa7d9570700 1 -- 172.21.15.19:0/417672903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7c8011a20 con 0x7fa7d4109af0 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.322+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/417672903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 msgr2=0x7fa7d4109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.322+0000 7fa7dc7d6700 1 --2- 172.21.15.19:0/417672903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d4109f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80098a0 tx=0x7fa7c8009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.323+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/417672903 shutdown_connections 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.323+0000 7fa7dc7d6700 1 --2- 172.21.15.19:0/417672903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d4109f00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.323+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/417672903 >> 172.21.15.19:0/417672903 conn(0x7fa7d4105110 msgr2=0x7fa7d41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.323+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/417672903 shutdown_connections 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.323+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/417672903 wait complete. 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.323+0000 7fa7dc7d6700 1 Processor -- start 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.323+0000 7fa7dc7d6700 1 -- start start 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7dc7d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7dc7d6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d410e070 con 0x7fa7d4109af0 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7da572700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7da572700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58250/0 (socket says 172.21.15.19:58250) 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7da572700 1 -- 172.21.15.19:0/1953819126 learned_addr learned my addr 172.21.15.19:0/1953819126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7da572700 1 -- 172.21.15.19:0/1953819126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d419e4d0 con 0x7fa7d4109af0 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7da572700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d419df90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009130 tx=0x7fa7c8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:33.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7c8003aa0 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c801a8e0 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d419e760 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.324+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d41a1370 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.325+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7c80248f0 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.325+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7fa7c8022020 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.326+0000 7fa7c77fe700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa7c003d4a0 0x7fa7c003f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.326+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa7c8053900 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.326+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d4064cf0 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.326+0000 7fa7d9d71700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa7c003d4a0 0x7fa7c003f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.326+0000 7fa7d9d71700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa7c003d4a0 0x7fa7c003f9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0006fd0 tx=0x7fa7d0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.331+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7c802c350 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.515+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7fa7c8018350 con 0x7fa7d4109af0 2024-04-25T10:45:33.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.564+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fa7d4106160 con 0x7fa7c003d4a0 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.682+0000 7fa7c77fe700 1 -- 172.21.15.19:0/1953819126 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fa7d4106160 con 0x7fa7c003d4a0 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa7c003d4a0 msgr2=0x7fa7c003f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa7c003d4a0 0x7fa7c003f9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0006fd0 tx=0x7fa7d0006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 msgr2=0x7fa7d419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d419df90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009130 tx=0x7fa7c8011aa0 comp rx=0 tx=0).stop 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 shutdown_connections 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa7c003d4a0 0x7fa7c003f9b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 --2- 172.21.15.19:0/1953819126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7d4109af0 0x7fa7d419df90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 >> 172.21.15.19:0/1953819126 conn(0x7fa7d4105110 msgr2=0x7fa7d4105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 shutdown_connections 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:32.685+0000 7fa7dc7d6700 1 -- 172.21.15.19:0/1953819126 wait complete. 2024-04-25T10:45:33.254 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-04-25T10:45:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:33 smithi019 ceph-mon[29209]: mgrmap e12: x(active, since 2s) 2024-04-25T10:45:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:33 smithi019 ceph-mon[29209]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:35.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$zPWBg.ms43HFCmQkPBpkYe7GpmaDfnaVRJaKt4mzsSgbzJI3W2UIu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714041934, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.646+0000 7f0d1dfa4700 1 Processor -- start 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.646+0000 7f0d1dfa4700 1 -- start start 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.646+0000 7f0d1dfa4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.646+0000 7f0d1dfa4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1807ce20 con 0x7f0d181083d0 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.646+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.646+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56042/0 (socket says 172.21.15.19:56042) 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.646+0000 7f0d177fe700 1 -- 172.21.15.19:0/1227996302 learned_addr learned my addr 172.21.15.19:0/1227996302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.647+0000 7f0d177fe700 1 -- 172.21.15.19:0/1227996302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d18108d20 con 0x7f0d181083d0 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.647+0000 7f0d177fe700 1 --2- 172.21.15.19:0/1227996302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181087e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d080098a0 tx=0x7f0d08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45dc5e37f625aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.647+0000 7f0d167fc700 1 -- 172.21.15.19:0/1227996302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08004160 con 0x7f0d181083d0 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.647+0000 7f0d167fc700 1 -- 172.21.15.19:0/1227996302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d08004300 con 0x7f0d181083d0 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.647+0000 7f0d167fc700 1 -- 172.21.15.19:0/1227996302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08011a20 con 0x7f0d181083d0 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.648+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1227996302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 msgr2=0x7f0d181087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.648+0000 7f0d1dfa4700 1 --2- 172.21.15.19:0/1227996302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181087e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d080098a0 tx=0x7f0d08009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.648+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1227996302 shutdown_connections 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.648+0000 7f0d1dfa4700 1 --2- 172.21.15.19:0/1227996302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181087e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.648+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1227996302 >> 172.21.15.19:0/1227996302 conn(0x7f0d18104040 msgr2=0x7f0d18106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.648+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1227996302 shutdown_connections 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.648+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1227996302 wait complete. 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.649+0000 7f0d1dfa4700 1 Processor -- start 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.649+0000 7f0d1dfa4700 1 -- start start 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.650+0000 7f0d1dfa4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.650+0000 7f0d1dfa4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18109de0 con 0x7f0d181083d0 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.650+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.650+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56054/0 (socket says 172.21.15.19:56054) 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.650+0000 7f0d177fe700 1 -- 172.21.15.19:0/1954550567 learned_addr learned my addr 172.21.15.19:0/1954550567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.650+0000 7f0d177fe700 1 -- 172.21.15.19:0/1954550567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d181a0620 con 0x7f0d181083d0 2024-04-25T10:45:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.650+0000 7f0d177fe700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181a00e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009520 tx=0x7f0d08011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.651+0000 7f0d14ff9700 1 -- 172.21.15.19:0/1954550567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d080039e0 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.651+0000 7f0d14ff9700 1 -- 172.21.15.19:0/1954550567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0801b8f0 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.651+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d181a08b0 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.651+0000 7f0d14ff9700 1 -- 172.21.15.19:0/1954550567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d080197d0 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.651+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d181a34c0 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.652+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d18064cf0 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.652+0000 7f0d14ff9700 1 -- 172.21.15.19:0/1954550567 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f0d08019a40 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.652+0000 7f0d14ff9700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0d0003d620 0x7f0d0003fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.652+0000 7f0d14ff9700 1 -- 172.21.15.19:0/1954550567 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0d08052f60 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.652+0000 7f0d16ffd700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0d0003d620 0x7f0d0003fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.653+0000 7f0d16ffd700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0d0003d620 0x7f0d0003fb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c006fd0 tx=0x7f0d0c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.658+0000 7f0d14ff9700 1 -- 172.21.15.19:0/1954550567 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d0801f030 con 0x7f0d181083d0 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:33.886+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f0d1802e780 con 0x7f0d0003d620 2024-04-25T10:45:35.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.133+0000 7f0d14ff9700 1 -- 172.21.15.19:0/1954550567 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f0d1802e780 con 0x7f0d0003d620 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0d0003d620 msgr2=0x7f0d0003fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0d0003d620 0x7f0d0003fb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c006fd0 tx=0x7f0d0c006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 msgr2=0x7f0d181a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181a00e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009520 tx=0x7f0d08011aa0 comp rx=0 tx=0).stop 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 shutdown_connections 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0d0003d620 0x7f0d0003fb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 --2- 172.21.15.19:0/1954550567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181083d0 0x7f0d181a00e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 >> 172.21.15.19:0/1954550567 conn(0x7f0d18104040 msgr2=0x7f0d18194060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 shutdown_connections 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:34.135+0000 7f0d1dfa4700 1 -- 172.21.15.19:0/1954550567 wait complete. 2024-04-25T10:45:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-04-25T10:45:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:35 smithi019 ceph-mon[29209]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:35 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:36.228 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:36 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/716840344' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.704+0000 7f0199d6d700 1 Processor -- start 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.704+0000 7f0199d6d700 1 -- start start 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.704+0000 7f0199d6d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019410ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.704+0000 7f0199d6d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f019407d730 con 0x7f019410a5f0 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.704+0000 7f01937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019410ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.704+0000 7f01937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019410ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56056/0 (socket says 172.21.15.19:56056) 2024-04-25T10:45:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.704+0000 7f01937fe700 1 -- 172.21.15.19:0/2326797000 learned_addr learned my addr 172.21.15.19:0/2326797000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.705+0000 7f01937fe700 1 -- 172.21.15.19:0/2326797000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019410cf50 con 0x7f019410a5f0 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.705+0000 7f01937fe700 1 --2- 172.21.15.19:0/2326797000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019410ca10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01840098a0 tx=0x7f0184009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117e910752cd1644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.705+0000 7f01927fc700 1 -- 172.21.15.19:0/2326797000 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0184004160 con 0x7f019410a5f0 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.705+0000 7f01927fc700 1 -- 172.21.15.19:0/2326797000 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0184004300 con 0x7f019410a5f0 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.705+0000 7f01927fc700 1 -- 172.21.15.19:0/2326797000 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0184011a20 con 0x7f019410a5f0 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 -- 172.21.15.19:0/2326797000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 msgr2=0x7f019410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 --2- 172.21.15.19:0/2326797000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019410ca10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01840098a0 tx=0x7f0184009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 -- 172.21.15.19:0/2326797000 shutdown_connections 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 --2- 172.21.15.19:0/2326797000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019410ca10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 -- 172.21.15.19:0/2326797000 >> 172.21.15.19:0/2326797000 conn(0x7f0194104090 msgr2=0x7f0194106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 -- 172.21.15.19:0/2326797000 shutdown_connections 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 -- 172.21.15.19:0/2326797000 wait complete. 2024-04-25T10:45:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.706+0000 7f0199d6d700 1 Processor -- start 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f0199d6d700 1 -- start start 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f0199d6d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f0199d6d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f019410df50 con 0x7f019410a5f0 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f01937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f01937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56064/0 (socket says 172.21.15.19:56064) 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f01937fe700 1 -- 172.21.15.19:0/716840344 learned_addr learned my addr 172.21.15.19:0/716840344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f01937fe700 1 -- 172.21.15.19:0/716840344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019419e3f0 con 0x7f019410a5f0 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.707+0000 7f01937fe700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019419deb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f0184011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.708+0000 7f0190ff9700 1 -- 172.21.15.19:0/716840344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0184003aa0 con 0x7f019410a5f0 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.708+0000 7f0190ff9700 1 -- 172.21.15.19:0/716840344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f018401b8f0 con 0x7f019410a5f0 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.708+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019419e680 con 0x7f019410a5f0 2024-04-25T10:45:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.708+0000 7f0190ff9700 1 -- 172.21.15.19:0/716840344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0184023890 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.708+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941a1290 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.709+0000 7f0190ff9700 1 -- 172.21.15.19:0/716840344 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f018402c3f0 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.709+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0194064cf0 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.709+0000 7f0190ff9700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f017c03d620 0x7f017c03fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.709+0000 7f0190ff9700 1 -- 172.21.15.19:0/716840344 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0184052f20 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.709+0000 7f0192ffd700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f017c03d620 0x7f017c03fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.710+0000 7f0192ffd700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f017c03d620 0x7f017c03fb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0188006fd0 tx=0x7f0188006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.714+0000 7f0190ff9700 1 -- 172.21.15.19:0/716840344 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f018401f030 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.923+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f01941a1460 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.924+0000 7f0190ff9700 1 -- 172.21.15.19:0/716840344 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f018401bda0 con 0x7f019410a5f0 2024-04-25T10:45:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f017c03d620 msgr2=0x7f017c03fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f017c03d620 0x7f017c03fb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0188006fd0 tx=0x7f0188006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 msgr2=0x7f019419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019419deb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f0184011aa0 comp rx=0 tx=0).stop 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 shutdown_connections 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f017c03d620 0x7f017c03fb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 --2- 172.21.15.19:0/716840344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f019410a5f0 0x7f019419deb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 >> 172.21.15.19:0/716840344 conn(0x7f0194104090 msgr2=0x7f01941048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 shutdown_connections 2024-04-25T10:45:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:35.926+0000 7f0199d6d700 1 -- 172.21.15.19:0/716840344 wait complete. 2024-04-25T10:45:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:45:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T10:45:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:45:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T10:45:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T10:45:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T10:45:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-04-25T10:45:36.495 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019:8443/ 2024-04-25T10:45:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-04-25T10:45:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: Password: ufz3h1d3rf 2024-04-25T10:45:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config directory 2024-04-25T10:45:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-04-25T10:45:37.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:37 smithi019 ceph-mon[29209]: mgrmap e13: x(active, since 6s) 2024-04-25T10:45:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.905+0000 7f97eb6ca700 1 Processor -- start 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.905+0000 7f97eb6ca700 1 -- start start 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.905+0000 7f97eb6ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e410ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.905+0000 7f97eb6ca700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97e407d730 con 0x7f97e410a5f0 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.905+0000 7f97e9466700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e410ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.905+0000 7f97e9466700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e410ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56068/0 (socket says 172.21.15.19:56068) 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.905+0000 7f97e9466700 1 -- 172.21.15.19:0/3526329005 learned_addr learned my addr 172.21.15.19:0/3526329005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.906+0000 7f97e9466700 1 -- 172.21.15.19:0/3526329005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e410cf50 con 0x7f97e410a5f0 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.906+0000 7f97e9466700 1 --2- 172.21.15.19:0/3526329005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e410ca10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97e00098a0 tx=0x7f97e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6681b09efee01c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.906+0000 7f97dbfff700 1 -- 172.21.15.19:0/3526329005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e0004160 con 0x7f97e410a5f0 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.906+0000 7f97dbfff700 1 -- 172.21.15.19:0/3526329005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e0004300 con 0x7f97e410a5f0 2024-04-25T10:45:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.906+0000 7f97dbfff700 1 -- 172.21.15.19:0/3526329005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e0011a20 con 0x7f97e410a5f0 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.907+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3526329005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 msgr2=0x7f97e410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.907+0000 7f97eb6ca700 1 --2- 172.21.15.19:0/3526329005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e410ca10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97e00098a0 tx=0x7f97e0009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.907+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3526329005 shutdown_connections 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.907+0000 7f97eb6ca700 1 --2- 172.21.15.19:0/3526329005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e410ca10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.907+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3526329005 >> 172.21.15.19:0/3526329005 conn(0x7f97e4104090 msgr2=0x7f97e4106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.907+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3526329005 shutdown_connections 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.907+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3526329005 wait complete. 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97eb6ca700 1 Processor -- start 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97eb6ca700 1 -- start start 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97eb6ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97eb6ca700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97e410df50 con 0x7f97e410a5f0 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97e9466700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97e9466700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56084/0 (socket says 172.21.15.19:56084) 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97e9466700 1 -- 172.21.15.19:0/3076619757 learned_addr learned my addr 172.21.15.19:0/3076619757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.908+0000 7f97e9466700 1 -- 172.21.15.19:0/3076619757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e419e3f0 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.909+0000 7f97e9466700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e419deb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f97e00119f0 tx=0x7f97e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.909+0000 7f97da7fc700 1 -- 172.21.15.19:0/3076619757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e00039e0 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.909+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e419e680 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.909+0000 7f97da7fc700 1 -- 172.21.15.19:0/3076619757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e001b8f0 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.909+0000 7f97da7fc700 1 -- 172.21.15.19:0/3076619757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e00237a0 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.909+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e41a1290 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.910+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e4064cf0 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.910+0000 7f97da7fc700 1 -- 172.21.15.19:0/3076619757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f97e0023aa0 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.910+0000 7f97da7fc700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f97d003d610 0x7f97d003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.910+0000 7f97da7fc700 1 -- 172.21.15.19:0/3076619757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f97e0052e50 con 0x7f97e410a5f0 2024-04-25T10:45:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.910+0000 7f97e8c65700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f97d003d610 0x7f97d003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.911+0000 7f97e8c65700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f97d003d610 0x7f97d003fb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f97d4006fd0 tx=0x7f97d4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:36.915+0000 7f97da7fc700 1 -- 172.21.15.19:0/3076619757 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97e001f030 con 0x7f97e410a5f0 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.120+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f97e41a1460 con 0x7f97e410a5f0 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.121+0000 7f97da7fc700 1 -- 172.21.15.19:0/3076619757 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f97e0017e90 con 0x7f97e410a5f0 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f97d003d610 msgr2=0x7f97d003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f97d003d610 0x7f97d003fb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f97d4006fd0 tx=0x7f97d4006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 msgr2=0x7f97e419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e419deb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f97e00119f0 tx=0x7f97e0011ad0 comp rx=0 tx=0).stop 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 shutdown_connections 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f97d003d610 0x7f97d003fb20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 --2- 172.21.15.19:0/3076619757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97e410a5f0 0x7f97e419deb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 >> 172.21.15.19:0/3076619757 conn(0x7f97e4104090 msgr2=0x7f97e41048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 shutdown_connections 2024-04-25T10:45:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:37.123+0000 7f97eb6ca700 1 -- 172.21.15.19:0/3076619757 wait complete. 2024-04-25T10:45:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.159+0000 7f24e43de700 1 Processor -- start 2024-04-25T10:45:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.159+0000 7f24e43de700 1 -- start start 2024-04-25T10:45:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.159+0000 7f24e43de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.159+0000 7f24e43de700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc10a4a0 con 0x7f24dc109aa0 2024-04-25T10:45:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.159+0000 7f24e217a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.159+0000 7f24e217a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc109eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56088/0 (socket says 172.21.15.19:56088) 2024-04-25T10:45:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.159+0000 7f24e217a700 1 -- 172.21.15.19:0/3288629198 learned_addr learned my addr 172.21.15.19:0/3288629198 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.160+0000 7f24e217a700 1 -- 172.21.15.19:0/3288629198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc10ac70 con 0x7f24dc109aa0 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.160+0000 7f24e217a700 1 --2- 172.21.15.19:0/3288629198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc109eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24d80098a0 tx=0x7f24d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de783f17d731569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.160+0000 7f24e1178700 1 -- 172.21.15.19:0/3288629198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d8004160 con 0x7f24dc109aa0 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.160+0000 7f24e1178700 1 -- 172.21.15.19:0/3288629198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d8004300 con 0x7f24dc109aa0 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.160+0000 7f24e1178700 1 -- 172.21.15.19:0/3288629198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d8011a20 con 0x7f24dc109aa0 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.161+0000 7f24e43de700 1 -- 172.21.15.19:0/3288629198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 msgr2=0x7f24dc109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.161+0000 7f24e43de700 1 --2- 172.21.15.19:0/3288629198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc109eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24d80098a0 tx=0x7f24d8009bb0 comp rx=0 tx=0).stop 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.161+0000 7f24e43de700 1 -- 172.21.15.19:0/3288629198 shutdown_connections 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.161+0000 7f24e43de700 1 --2- 172.21.15.19:0/3288629198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc109eb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.161+0000 7f24e43de700 1 -- 172.21.15.19:0/3288629198 >> 172.21.15.19:0/3288629198 conn(0x7f24dc1050c0 msgr2=0x7f24dc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.161+0000 7f24e43de700 1 -- 172.21.15.19:0/3288629198 shutdown_connections 2024-04-25T10:45:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.161+0000 7f24e43de700 1 -- 172.21.15.19:0/3288629198 wait complete. 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e43de700 1 Processor -- start 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e43de700 1 -- start start 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e43de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc07ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e43de700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc10e020 con 0x7f24dc109aa0 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e217a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc07ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e217a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc07ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56090/0 (socket says 172.21.15.19:56090) 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e217a700 1 -- 172.21.15.19:0/3104287408 learned_addr learned my addr 172.21.15.19:0/3104287408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.162+0000 7f24e217a700 1 -- 172.21.15.19:0/3104287408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc07efd0 con 0x7f24dc109aa0 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.163+0000 7f24e217a700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc07ea90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009130 tx=0x7f24d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.163+0000 7f24d37fe700 1 -- 172.21.15.19:0/3104287408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d8003aa0 con 0x7f24dc109aa0 2024-04-25T10:45:39.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.163+0000 7f24d37fe700 1 -- 172.21.15.19:0/3104287408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d801a8e0 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.163+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc07d220 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.163+0000 7f24d37fe700 1 -- 172.21.15.19:0/3104287408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24d80248f0 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.163+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc07d700 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.164+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc064cf0 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.164+0000 7f24d37fe700 1 -- 172.21.15.19:0/3104287408 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f24d8022020 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.164+0000 7f24d37fe700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24c803d6c0 0x7f24c803fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.164+0000 7f24d37fe700 1 -- 172.21.15.19:0/3104287408 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24d8052870 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.164+0000 7f24e1979700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24c803d6c0 0x7f24c803fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.165+0000 7f24e1979700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24c803d6c0 0x7f24c803fbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006fd0 tx=0x7f24cc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.169+0000 7f24d37fe700 1 -- 172.21.15.19:0/3104287408 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24d80172b0 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.492+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f24dc07ddb0 con 0x7f24dc109aa0 2024-04-25T10:45:39.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.495+0000 7f24d37fe700 1 -- 172.21.15.19:0/3104287408 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f24d800fdd0 con 0x7f24dc109aa0 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24c803d6c0 msgr2=0x7f24c803fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24c803d6c0 0x7f24c803fbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006fd0 tx=0x7f24cc006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 msgr2=0x7f24dc07ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc07ea90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009130 tx=0x7f24d8011aa0 comp rx=0 tx=0).stop 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 shutdown_connections 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24c803d6c0 0x7f24c803fbd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 --2- 172.21.15.19:0/3104287408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24dc109aa0 0x7f24dc07ea90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 >> 172.21.15.19:0/3104287408 conn(0x7f24dc1050c0 msgr2=0x7f24dc1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 shutdown_connections 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T10:45:38.497+0000 7f24e43de700 1 -- 172.21.15.19:0/3104287408 wait complete. 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T10:45:39.072 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-04-25T10:45:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T10:45:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:39.074 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-04-25T10:45:39.109 INFO:tasks.cephadm:Fetching config... 2024-04-25T10:45:39.109 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:45:39.109 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T10:45:39.126 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T10:45:39.126 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:45:39.127 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T10:45:39.182 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T10:45:39.183 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:45:39.183 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T10:45:39.248 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T10:45:39.248 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:45:39.248 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T10:45:39.304 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T10:45:39.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5TcSToHXiFVXiYzVcuvkcLgItB3dWrb1V0rJG4MZIzJGxLOJmSo6LNENmNtMf1+p0lz2uT2Vi79zSz0P3MlGDGw1nZsnbyyKP4OZeMA8Kb1tAuuFj3U4rLgN2UNpYEZIOTLb5zPNaMeK8TQ6XYXuBfLYCISfuGpk7uFZFPZr449diyQusEB/5cFW+hSxhS+VlKb2cP6dhCNLFdcQZrpWcrFgSPadPhI5vsPpi61K73enA60F/C15fuoC6z2D+u/QoV7z3ztJ+CGBw1J0amkrLLHRbxHhUv/mkW+GpzKAArNNet+bfvf/rrJ11Lzz/U1a56jYfP5/B89tFTviLpnUVeuyGcravpvr+ar5CmQduzXd4tWmbJW6VSs6PsrTwa9jZMcc/nHzxdokvWNjjX4jv8dgEdWQqfGi6tlWW6w8c/HNRIKzSDRqM8okqph9KV/5S0PJ+K+0i6o0fGmQ6cT2cH6LBgJ4CGcv+m88+RisBK01jff8nvduwaS1+Rc8NufU= ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T10:45:39.391 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5TcSToHXiFVXiYzVcuvkcLgItB3dWrb1V0rJG4MZIzJGxLOJmSo6LNENmNtMf1+p0lz2uT2Vi79zSz0P3MlGDGw1nZsnbyyKP4OZeMA8Kb1tAuuFj3U4rLgN2UNpYEZIOTLb5zPNaMeK8TQ6XYXuBfLYCISfuGpk7uFZFPZr449diyQusEB/5cFW+hSxhS+VlKb2cP6dhCNLFdcQZrpWcrFgSPadPhI5vsPpi61K73enA60F/C15fuoC6z2D+u/QoV7z3ztJ+CGBw1J0amkrLLHRbxHhUv/mkW+GpzKAArNNet+bfvf/rrJ11Lzz/U1a56jYfP5/B89tFTviLpnUVeuyGcravpvr+ar5CmQduzXd4tWmbJW6VSs6PsrTwa9jZMcc/nHzxdokvWNjjX4jv8dgEdWQqfGi6tlWW6w8c/HNRIKzSDRqM8okqph9KV/5S0PJ+K+0i6o0fGmQ6cT2cH6LBgJ4CGcv+m88+RisBK01jff8nvduwaS1+Rc8NufU= ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:45:39.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5TcSToHXiFVXiYzVcuvkcLgItB3dWrb1V0rJG4MZIzJGxLOJmSo6LNENmNtMf1+p0lz2uT2Vi79zSz0P3MlGDGw1nZsnbyyKP4OZeMA8Kb1tAuuFj3U4rLgN2UNpYEZIOTLb5zPNaMeK8TQ6XYXuBfLYCISfuGpk7uFZFPZr449diyQusEB/5cFW+hSxhS+VlKb2cP6dhCNLFdcQZrpWcrFgSPadPhI5vsPpi61K73enA60F/C15fuoC6z2D+u/QoV7z3ztJ+CGBw1J0amkrLLHRbxHhUv/mkW+GpzKAArNNet+bfvf/rrJ11Lzz/U1a56jYfP5/B89tFTviLpnUVeuyGcravpvr+ar5CmQduzXd4tWmbJW6VSs6PsrTwa9jZMcc/nHzxdokvWNjjX4jv8dgEdWQqfGi6tlWW6w8c/HNRIKzSDRqM8okqph9KV/5S0PJ+K+0i6o0fGmQ6cT2cH6LBgJ4CGcv+m88+RisBK01jff8nvduwaS1+Rc8NufU= ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T10:45:39.441 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5TcSToHXiFVXiYzVcuvkcLgItB3dWrb1V0rJG4MZIzJGxLOJmSo6LNENmNtMf1+p0lz2uT2Vi79zSz0P3MlGDGw1nZsnbyyKP4OZeMA8Kb1tAuuFj3U4rLgN2UNpYEZIOTLb5zPNaMeK8TQ6XYXuBfLYCISfuGpk7uFZFPZr449diyQusEB/5cFW+hSxhS+VlKb2cP6dhCNLFdcQZrpWcrFgSPadPhI5vsPpi61K73enA60F/C15fuoC6z2D+u/QoV7z3ztJ+CGBw1J0amkrLLHRbxHhUv/mkW+GpzKAArNNet+bfvf/rrJ11Lzz/U1a56jYfP5/B89tFTviLpnUVeuyGcravpvr+ar5CmQduzXd4tWmbJW6VSs6PsrTwa9jZMcc/nHzxdokvWNjjX4jv8dgEdWQqfGi6tlWW6w8c/HNRIKzSDRqM8okqph9KV/5S0PJ+K+0i6o0fGmQ6cT2cH6LBgJ4CGcv+m88+RisBK01jff8nvduwaS1+Rc8NufU= ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:45:39.453 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5TcSToHXiFVXiYzVcuvkcLgItB3dWrb1V0rJG4MZIzJGxLOJmSo6LNENmNtMf1+p0lz2uT2Vi79zSz0P3MlGDGw1nZsnbyyKP4OZeMA8Kb1tAuuFj3U4rLgN2UNpYEZIOTLb5zPNaMeK8TQ6XYXuBfLYCISfuGpk7uFZFPZr449diyQusEB/5cFW+hSxhS+VlKb2cP6dhCNLFdcQZrpWcrFgSPadPhI5vsPpi61K73enA60F/C15fuoC6z2D+u/QoV7z3ztJ+CGBw1J0amkrLLHRbxHhUv/mkW+GpzKAArNNet+bfvf/rrJ11Lzz/U1a56jYfP5/B89tFTviLpnUVeuyGcravpvr+ar5CmQduzXd4tWmbJW6VSs6PsrTwa9jZMcc/nHzxdokvWNjjX4jv8dgEdWQqfGi6tlWW6w8c/HNRIKzSDRqM8okqph9KV/5S0PJ+K+0i6o0fGmQ6cT2cH6LBgJ4CGcv+m88+RisBK01jff8nvduwaS1+Rc8NufU= ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T10:45:39.492 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5TcSToHXiFVXiYzVcuvkcLgItB3dWrb1V0rJG4MZIzJGxLOJmSo6LNENmNtMf1+p0lz2uT2Vi79zSz0P3MlGDGw1nZsnbyyKP4OZeMA8Kb1tAuuFj3U4rLgN2UNpYEZIOTLb5zPNaMeK8TQ6XYXuBfLYCISfuGpk7uFZFPZr449diyQusEB/5cFW+hSxhS+VlKb2cP6dhCNLFdcQZrpWcrFgSPadPhI5vsPpi61K73enA60F/C15fuoC6z2D+u/QoV7z3ztJ+CGBw1J0amkrLLHRbxHhUv/mkW+GpzKAArNNet+bfvf/rrJ11Lzz/U1a56jYfP5/B89tFTviLpnUVeuyGcravpvr+ar5CmQduzXd4tWmbJW6VSs6PsrTwa9jZMcc/nHzxdokvWNjjX4jv8dgEdWQqfGi6tlWW6w8c/HNRIKzSDRqM8okqph9KV/5S0PJ+K+0i6o0fGmQ6cT2cH6LBgJ4CGcv+m88+RisBK01jff8nvduwaS1+Rc8NufU= ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:45:39.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T10:45:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:45:39.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:39 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3104287408' entity='client.admin' 2024-04-25T10:45:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.901+0000 7f2e14e5c700 1 -- 172.21.15.19:0/2461564786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 msgr2=0x7f2e10102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.901+0000 7f2e14e5c700 1 --2- 172.21.15.19:0/2461564786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10102d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009910 tx=0x7f2e00009c20 comp rx=0 tx=0).stop 2024-04-25T10:45:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.901+0000 7f2e14e5c700 1 -- 172.21.15.19:0/2461564786 shutdown_connections 2024-04-25T10:45:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.901+0000 7f2e14e5c700 1 --2- 172.21.15.19:0/2461564786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10102d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.901+0000 7f2e14e5c700 1 -- 172.21.15.19:0/2461564786 >> 172.21.15.19:0/2461564786 conn(0x7f2e100fdfe0 msgr2=0x7f2e10100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.901+0000 7f2e14e5c700 1 -- 172.21.15.19:0/2461564786 shutdown_connections 2024-04-25T10:45:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.901+0000 7f2e14e5c700 1 -- 172.21.15.19:0/2461564786 wait complete. 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e14e5c700 1 Processor -- start 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e14e5c700 1 -- start start 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e14e5c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e14e5c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10108940 con 0x7f2e10102980 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e0e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e0e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56108/0 (socket says 172.21.15.19:56108) 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e0e59c700 1 -- 172.21.15.19:0/1494318099 learned_addr learned my addr 172.21.15.19:0/1494318099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.902+0000 7f2e0e59c700 1 -- 172.21.15.19:0/1494318099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e101989d0 con 0x7f2e10102980 2024-04-25T10:45:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.903+0000 7f2e0e59c700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10198490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e000057d0 tx=0x7f2e00005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.903+0000 7f2dff7fe700 1 -- 172.21.15.19:0/1494318099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e00027070 con 0x7f2e10102980 2024-04-25T10:45:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.903+0000 7f2dff7fe700 1 -- 172.21.15.19:0/1494318099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e0000be10 con 0x7f2e10102980 2024-04-25T10:45:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.903+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e10198c60 con 0x7f2e10102980 2024-04-25T10:45:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.903+0000 7f2dff7fe700 1 -- 172.21.15.19:0/1494318099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e00022dd0 con 0x7f2e10102980 2024-04-25T10:45:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.903+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1019b870 con 0x7f2e10102980 2024-04-25T10:45:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.904+0000 7f2dff7fe700 1 -- 172.21.15.19:0/1494318099 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f2e0002a070 con 0x7f2e10102980 2024-04-25T10:45:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.904+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e101090c0 con 0x7f2e10102980 2024-04-25T10:45:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.904+0000 7f2dff7fe700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2df803d610 0x7f2df803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.904+0000 7f2dff7fe700 1 -- 172.21.15.19:0/1494318099 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e0001e030 con 0x7f2e10102980 2024-04-25T10:45:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.905+0000 7f2e0dd9b700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2df803d610 0x7f2df803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.905+0000 7f2e0dd9b700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2df803d610 0x7f2df803fb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2e04006fd0 tx=0x7f2e04006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:41.910+0000 7f2dff7fe700 1 -- 172.21.15.19:0/1494318099 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e00013070 con 0x7f2e10102980 2024-04-25T10:45:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.127+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f2e1002e620 con 0x7f2e10102980 2024-04-25T10:45:42.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.129+0000 7f2dff7fe700 1 -- 172.21.15.19:0/1494318099 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2e000292c0 con 0x7f2e10102980 2024-04-25T10:45:42.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2df803d610 msgr2=0x7f2df803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:42.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2df803d610 0x7f2df803fb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2e04006fd0 tx=0x7f2e04006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 msgr2=0x7f2e10198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10198490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e000057d0 tx=0x7f2e00005800 comp rx=0 tx=0).stop 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 shutdown_connections 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2df803d610 0x7f2df803fb20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 --2- 172.21.15.19:0/1494318099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e10102980 0x7f2e10198490 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 >> 172.21.15.19:0/1494318099 conn(0x7f2e100fdfe0 msgr2=0x7f2e100fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 shutdown_connections 2024-04-25T10:45:42.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:42.132+0000 7f2e14e5c700 1 -- 172.21.15.19:0/1494318099 wait complete. 2024-04-25T10:45:42.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:42.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:42.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:45:42.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:42.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:42.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:43.020 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T10:45:43.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T10:45:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:43 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1494318099' entity='client.admin' 2024-04-25T10:45:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:43.829 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:45:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.320+0000 7fb11588d700 1 -- 172.21.15.19:0/2943005044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 msgr2=0x7fb110103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.320+0000 7fb11588d700 1 --2- 172.21.15.19:0/2943005044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110103ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb100009910 tx=0x7fb100009c20 comp rx=0 tx=0).stop 2024-04-25T10:45:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.320+0000 7fb11588d700 1 -- 172.21.15.19:0/2943005044 shutdown_connections 2024-04-25T10:45:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.320+0000 7fb11588d700 1 --2- 172.21.15.19:0/2943005044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110103ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.320+0000 7fb11588d700 1 -- 172.21.15.19:0/2943005044 >> 172.21.15.19:0/2943005044 conn(0x7fb1100ff0d0 msgr2=0x7fb110101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.320+0000 7fb11588d700 1 -- 172.21.15.19:0/2943005044 shutdown_connections 2024-04-25T10:45:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.320+0000 7fb11588d700 1 -- 172.21.15.19:0/2943005044 wait complete. 2024-04-25T10:45:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb11588d700 1 Processor -- start 2024-04-25T10:45:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb11588d700 1 -- start start 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb11588d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb11588d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb110109a50 con 0x7fb110103a90 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb10effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb10effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58810/0 (socket says 172.21.15.19:58810) 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb10effd700 1 -- 172.21.15.19:0/4111901684 learned_addr learned my addr 172.21.15.19:0/4111901684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.321+0000 7fb10effd700 1 -- 172.21.15.19:0/4111901684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb110077790 con 0x7fb110103a90 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.322+0000 7fb10effd700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110077250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1000057d0 tx=0x7fb100005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.322+0000 7fb11488b700 1 -- 172.21.15.19:0/4111901684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb100027070 con 0x7fb110103a90 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.322+0000 7fb11488b700 1 -- 172.21.15.19:0/4111901684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb10000be10 con 0x7fb110103a90 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.322+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1100759e0 con 0x7fb110103a90 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.322+0000 7fb11488b700 1 -- 172.21.15.19:0/4111901684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb100022dd0 con 0x7fb110103a90 2024-04-25T10:45:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.322+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb110075ec0 con 0x7fb110103a90 2024-04-25T10:45:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.323+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11010a1d0 con 0x7fb110103a90 2024-04-25T10:45:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.323+0000 7fb11488b700 1 -- 172.21.15.19:0/4111901684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb10002a070 con 0x7fb110103a90 2024-04-25T10:45:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.323+0000 7fb11488b700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb0f803d5c0 0x7fb0f803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.323+0000 7fb11488b700 1 -- 172.21.15.19:0/4111901684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb10001e030 con 0x7fb110103a90 2024-04-25T10:45:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.323+0000 7fb10e7fc700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb0f803d5c0 0x7fb0f803fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.324+0000 7fb10e7fc700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb0f803d5c0 0x7fb0f803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb104006fd0 tx=0x7fb104006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:45.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.329+0000 7fb11488b700 1 -- 172.21.15.19:0/4111901684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb100013070 con 0x7fb110103a90 2024-04-25T10:45:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.539+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb1101923f0 con 0x7fb0f803d5c0 2024-04-25T10:45:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.543+0000 7fb11488b700 1 -- 172.21.15.19:0/4111901684 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb1101923f0 con 0x7fb0f803d5c0 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb0f803d5c0 msgr2=0x7fb0f803fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb0f803d5c0 0x7fb0f803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb104006fd0 tx=0x7fb104006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 msgr2=0x7fb110077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110077250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1000057d0 tx=0x7fb100005800 comp rx=0 tx=0).stop 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 shutdown_connections 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb0f803d5c0 0x7fb0f803fad0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 --2- 172.21.15.19:0/4111901684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb110103a90 0x7fb110077250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.545+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 >> 172.21.15.19:0/4111901684 conn(0x7fb1100ff0d0 msgr2=0x7fb1100ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.546+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 shutdown_connections 2024-04-25T10:45:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:45.546+0000 7fb11588d700 1 -- 172.21.15.19:0/4111901684 wait complete. 2024-04-25T10:45:46.284 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi112 2024-04-25T10:45:46.284 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:45:46.284 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.conf 2024-04-25T10:45:46.303 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:45:46.303 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:45:46.361 INFO:tasks.cephadm:Adding host smithi112 to orchestrator... 2024-04-25T10:45:46.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch host add smithi112 2024-04-25T10:45:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:46.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:47.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:47 smithi019 ceph-mon[29209]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:45:47.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:47 smithi019 ceph-mon[29209]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:45:47.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:47 smithi019 ceph-mon[29209]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:45:48.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.030+0000 7f83ac2b0700 1 -- 172.21.15.19:0/286266160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 msgr2=0x7f83a4108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:48.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.030+0000 7f83ac2b0700 1 --2- 172.21.15.19:0/286266160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4108d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f839c00b0a0 tx=0x7f839c00b3b0 comp rx=0 tx=0).stop 2024-04-25T10:45:48.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.030+0000 7f83ac2b0700 1 -- 172.21.15.19:0/286266160 shutdown_connections 2024-04-25T10:45:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.030+0000 7f83ac2b0700 1 --2- 172.21.15.19:0/286266160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4108d40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.030+0000 7f83ac2b0700 1 -- 172.21.15.19:0/286266160 >> 172.21.15.19:0/286266160 conn(0x7f83a40fe480 msgr2=0x7f83a41008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.030+0000 7f83ac2b0700 1 -- 172.21.15.19:0/286266160 shutdown_connections 2024-04-25T10:45:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.030+0000 7f83ac2b0700 1 -- 172.21.15.19:0/286266160 wait complete. 2024-04-25T10:45:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.031+0000 7f83ac2b0700 1 Processor -- start 2024-04-25T10:45:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.031+0000 7f83ac2b0700 1 -- start start 2024-04-25T10:45:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83ac2b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83ac2b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83a410ce80 con 0x7f83a4108930 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83aa04c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83aa04c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58814/0 (socket says 172.21.15.19:58814) 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83aa04c700 1 -- 172.21.15.19:0/1131086090 learned_addr learned my addr 172.21.15.19:0/1131086090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83aa04c700 1 -- 172.21.15.19:0/1131086090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a4199cb0 con 0x7f83a4108930 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83aa04c700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4199770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f839c015040 tx=0x7f839c009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83a37fe700 1 -- 172.21.15.19:0/1131086090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f839c021020 con 0x7f83a4108930 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.032+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a4199f40 con 0x7f83a4108930 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.033+0000 7f83a37fe700 1 -- 172.21.15.19:0/1131086090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f839c007970 con 0x7f83a4108930 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.033+0000 7f83a37fe700 1 -- 172.21.15.19:0/1131086090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f839c017040 con 0x7f83a4108930 2024-04-25T10:45:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.033+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a419dfa0 con 0x7f83a4108930 2024-04-25T10:45:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.033+0000 7f83a37fe700 1 -- 172.21.15.19:0/1131086090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f839c024020 con 0x7f83a4108930 2024-04-25T10:45:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.033+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a4104040 con 0x7f83a4108930 2024-04-25T10:45:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.034+0000 7f83a37fe700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f838c03d670 0x7f838c03fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.034+0000 7f83a37fe700 1 -- 172.21.15.19:0/1131086090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f839c00f050 con 0x7f83a4108930 2024-04-25T10:45:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.034+0000 7f83a984b700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f838c03d670 0x7f838c03fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.034+0000 7f83a984b700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f838c03d670 0x7f838c03fb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8398006fd0 tx=0x7f8398006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:48.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.039+0000 7f83a37fe700 1 -- 172.21.15.19:0/1131086090 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f839c02b020 con 0x7f83a4108930 2024-04-25T10:45:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:48.250+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}) -- 0x7f83a419aad0 con 0x7f838c03d670 2024-04-25T10:45:49.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:48 smithi019 ceph-mon[29209]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:50.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:49 smithi019 ceph-mon[29209]: Deploying cephadm binary to smithi112 2024-04-25T10:45:51.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:50 smithi019 ceph-mon[29209]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:45:52.455 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi112' with addr '172.21.15.112' 2024-04-25T10:45:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.453+0000 7f83a37fe700 1 -- 172.21.15.19:0/1131086090 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f83a419aad0 con 0x7f838c03d670 2024-04-25T10:45:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f838c03d670 msgr2=0x7f838c03fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f838c03d670 0x7f838c03fb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8398006fd0 tx=0x7f8398006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 msgr2=0x7f83a4199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4199770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f839c015040 tx=0x7f839c009790 comp rx=0 tx=0).stop 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 shutdown_connections 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f838c03d670 0x7f838c03fb80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 --2- 172.21.15.19:0/1131086090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a4108930 0x7f83a4199770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 >> 172.21.15.19:0/1131086090 conn(0x7f83a40fe480 msgr2=0x7f83a4106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 shutdown_connections 2024-04-25T10:45:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:52.456+0000 7f83ac2b0700 1 -- 172.21.15.19:0/1131086090 wait complete. 2024-04-25T10:45:53.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T10:45:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:45:53.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:53 smithi019 ceph-mon[29209]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:45:53.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:53.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:53 smithi019 ceph-mon[29209]: Added host smithi112 2024-04-25T10:45:53.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:45:53.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:54.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:54.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.817+0000 7ff9103bd700 1 -- 172.21.15.19:0/429014975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 msgr2=0x7ff908104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:54.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.817+0000 7ff9103bd700 1 --2- 172.21.15.19:0/429014975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff908104fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff904009910 tx=0x7ff904009c20 comp rx=0 tx=0).stop 2024-04-25T10:45:54.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.817+0000 7ff9103bd700 1 -- 172.21.15.19:0/429014975 shutdown_connections 2024-04-25T10:45:54.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.817+0000 7ff9103bd700 1 --2- 172.21.15.19:0/429014975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff908104fc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:54.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.817+0000 7ff9103bd700 1 -- 172.21.15.19:0/429014975 >> 172.21.15.19:0/429014975 conn(0x7ff9080fe440 msgr2=0x7ff908100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:54.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.817+0000 7ff9103bd700 1 -- 172.21.15.19:0/429014975 shutdown_connections 2024-04-25T10:45:54.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.817+0000 7ff9103bd700 1 -- 172.21.15.19:0/429014975 wait complete. 2024-04-25T10:45:54.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.818+0000 7ff9103bd700 1 Processor -- start 2024-04-25T10:45:54.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.818+0000 7ff9103bd700 1 -- start start 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.818+0000 7ff9103bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff9081028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.818+0000 7ff9103bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff90810dff0 con 0x7ff908104bb0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.818+0000 7ff90e159700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff9081028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.818+0000 7ff90e159700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff9081028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57458/0 (socket says 172.21.15.19:57458) 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.818+0000 7ff90e159700 1 -- 172.21.15.19:0/3507576350 learned_addr learned my addr 172.21.15.19:0/3507576350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.819+0000 7ff90e159700 1 -- 172.21.15.19:0/3507576350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9081046a0 con 0x7ff908104bb0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.819+0000 7ff90e159700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff9081028a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9040056d0 tx=0x7ff9040057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.819+0000 7ff8ff7fe700 1 -- 172.21.15.19:0/3507576350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff904027070 con 0x7ff908104bb0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.819+0000 7ff8ff7fe700 1 -- 172.21.15.19:0/3507576350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90400be10 con 0x7ff908104bb0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.819+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff908102de0 con 0x7ff908104bb0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.819+0000 7ff8ff7fe700 1 -- 172.21.15.19:0/3507576350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff904022d20 con 0x7ff908104bb0 2024-04-25T10:45:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.819+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff908103240 con 0x7ff908104bb0 2024-04-25T10:45:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.820+0000 7ff8ff7fe700 1 -- 172.21.15.19:0/3507576350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7ff90402a070 con 0x7ff908104bb0 2024-04-25T10:45:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.820+0000 7ff8ff7fe700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8f403d670 0x7ff8f403fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.820+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff90806c2b0 con 0x7ff908104bb0 2024-04-25T10:45:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.820+0000 7ff8ff7fe700 1 -- 172.21.15.19:0/3507576350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff90401e030 con 0x7ff908104bb0 2024-04-25T10:45:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.820+0000 7ff90d958700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8f403d670 0x7ff8f403fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.821+0000 7ff90d958700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8f403d670 0x7ff8f403fb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8006fd0 tx=0x7ff8f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:54.826+0000 7ff8ff7fe700 1 -- 172.21.15.19:0/3507576350 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff904031020 con 0x7ff908104bb0 2024-04-25T10:45:55.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.038+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff908103d80 con 0x7ff8f403d670 2024-04-25T10:45:55.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.039+0000 7ff8ff7fe700 1 -- 172.21.15.19:0/3507576350 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7ff908103d80 con 0x7ff8f403d670 2024-04-25T10:45:55.041 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:45:55.041 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}] 2024-04-25T10:45:55.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8f403d670 msgr2=0x7ff8f403fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:55.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8f403d670 0x7ff8f403fb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8006fd0 tx=0x7ff8f8006e40 comp rx=0 tx=0).stop 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 msgr2=0x7ff9081028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff9081028a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9040056d0 tx=0x7ff9040057b0 comp rx=0 tx=0).stop 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 shutdown_connections 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8f403d670 0x7ff8f403fb80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 --2- 172.21.15.19:0/3507576350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff908104bb0 0x7ff9081028a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 >> 172.21.15.19:0/3507576350 conn(0x7ff9080fe440 msgr2=0x7ff908109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 shutdown_connections 2024-04-25T10:45:55.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:55.042+0000 7ff9103bd700 1 -- 172.21.15.19:0/3507576350 wait complete. 2024-04-25T10:45:55.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:55 smithi019 ceph-mon[29209]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:45:55.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:55.651 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-04-25T10:45:55.651 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:45:55.651 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-04-25T10:45:55.669 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:45:55.669 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:45:55.726 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-04-25T10:45:55.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch host add smithi169 2024-04-25T10:45:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:45:56.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:56 smithi019 ceph-mon[29209]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:45:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.487+0000 7f461179d700 1 -- 172.21.15.19:0/284671285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 msgr2=0x7f460c104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.487+0000 7f461179d700 1 --2- 172.21.15.19:0/284671285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c104fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009910 tx=0x7f45fc009c20 comp rx=0 tx=0).stop 2024-04-25T10:45:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.487+0000 7f461179d700 1 -- 172.21.15.19:0/284671285 shutdown_connections 2024-04-25T10:45:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.487+0000 7f461179d700 1 --2- 172.21.15.19:0/284671285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c104fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.487+0000 7f461179d700 1 -- 172.21.15.19:0/284671285 >> 172.21.15.19:0/284671285 conn(0x7f460c0fe440 msgr2=0x7f460c100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.487+0000 7f461179d700 1 -- 172.21.15.19:0/284671285 shutdown_connections 2024-04-25T10:45:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.487+0000 7f461179d700 1 -- 172.21.15.19:0/284671285 wait complete. 2024-04-25T10:45:57.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f461179d700 1 Processor -- start 2024-04-25T10:45:57.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f461179d700 1 -- start start 2024-04-25T10:45:57.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f461179d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:57.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f461179d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f460c10df50 con 0x7f460c104bb0 2024-04-25T10:45:57.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f460affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:57.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f460affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c1027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57478/0 (socket says 172.21.15.19:57478) 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f460affd700 1 -- 172.21.15.19:0/427843050 learned_addr learned my addr 172.21.15.19:0/427843050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.488+0000 7f460affd700 1 -- 172.21.15.19:0/427843050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c104630 con 0x7f460c104bb0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.489+0000 7f460affd700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c1027e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0056d0 tx=0x7f45fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.489+0000 7f45f3fff700 1 -- 172.21.15.19:0/427843050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45fc026070 con 0x7f460c104bb0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.489+0000 7f45f3fff700 1 -- 172.21.15.19:0/427843050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45fc00bba0 con 0x7f460c104bb0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.489+0000 7f45f3fff700 1 -- 172.21.15.19:0/427843050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45fc003920 con 0x7f460c104bb0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.489+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460c102d20 con 0x7f460c104bb0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.489+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460c1031e0 con 0x7f460c104bb0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.490+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460c06bfd0 con 0x7f460c104bb0 2024-04-25T10:45:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.490+0000 7f45f3fff700 1 -- 172.21.15.19:0/427843050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f45fc029070 con 0x7f460c104bb0 2024-04-25T10:45:57.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.490+0000 7f45f3fff700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f45f403d670 0x7f45f403fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:57.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.490+0000 7f45f3fff700 1 -- 172.21.15.19:0/427843050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f45fc01e070 con 0x7f460c104bb0 2024-04-25T10:45:57.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.490+0000 7f460a7fc700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f45f403d670 0x7f45f403fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:57.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.491+0000 7f460a7fc700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f45f403d670 0x7f45f403fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4600006fd0 tx=0x7f4600006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:57.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.495+0000 7f45f3fff700 1 -- 172.21.15.19:0/427843050 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45fc013070 con 0x7f460c104bb0 2024-04-25T10:45:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:45:57.708+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7f460c103db0 con 0x7f45f403d670 2024-04-25T10:45:57.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:57 smithi019 ceph-mon[29209]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:45:59.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: Deploying cephadm binary to smithi169 2024-04-25T10:45:59.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:45:59 smithi019 ceph-mon[29209]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:00 smithi019 ceph-mon[29209]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:46:01.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:01 smithi019 ceph-mon[29209]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.010+0000 7f45f3fff700 1 -- 172.21.15.19:0/427843050 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f460c103db0 con 0x7f45f403d670 2024-04-25T10:46:02.012 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-04-25T10:46:02.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f45f403d670 msgr2=0x7f45f403fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f45f403d670 0x7f45f403fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4600006fd0 tx=0x7f4600006e40 comp rx=0 tx=0).stop 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 msgr2=0x7f460c1027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c1027e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0056d0 tx=0x7f45fc0057b0 comp rx=0 tx=0).stop 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 shutdown_connections 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f45f403d670 0x7f45f403fb80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 --2- 172.21.15.19:0/427843050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460c104bb0 0x7f460c1027e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.012+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 >> 172.21.15.19:0/427843050 conn(0x7f460c0fe440 msgr2=0x7f460c105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.013+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 shutdown_connections 2024-04-25T10:46:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:02.013+0000 7f461179d700 1 -- 172.21.15.19:0/427843050 wait complete. 2024-04-25T10:46:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:02 smithi019 ceph-mon[29209]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:46:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:02.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T10:46:03.165 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:46:03.831 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:03 smithi019 ceph-mon[29209]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:03.832 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:03 smithi019 ceph-mon[29209]: Added host smithi169 2024-04-25T10:46:03.832 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:03 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.629+0000 7fd0253ee700 1 -- 172.21.15.19:0/3684000178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 msgr2=0x7fd020109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.629+0000 7fd0253ee700 1 --2- 172.21.15.19:0/3684000178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd020109e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd010009910 tx=0x7fd010009c20 comp rx=0 tx=0).stop 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.629+0000 7fd0253ee700 1 -- 172.21.15.19:0/3684000178 shutdown_connections 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.629+0000 7fd0253ee700 1 --2- 172.21.15.19:0/3684000178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd020109e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.629+0000 7fd0253ee700 1 -- 172.21.15.19:0/3684000178 >> 172.21.15.19:0/3684000178 conn(0x7fd0200ff560 msgr2=0x7fd0201019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.629+0000 7fd0253ee700 1 -- 172.21.15.19:0/3684000178 shutdown_connections 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.629+0000 7fd0253ee700 1 -- 172.21.15.19:0/3684000178 wait complete. 2024-04-25T10:46:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd0253ee700 1 Processor -- start 2024-04-25T10:46:04.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd0253ee700 1 -- start start 2024-04-25T10:46:04.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd0253ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd02019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:04.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd0253ee700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd02010df60 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd01effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd02019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd01effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd02019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41024/0 (socket says 172.21.15.19:41024) 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd01effd700 1 -- 172.21.15.19:0/852520728 learned_addr learned my addr 172.21.15.19:0/852520728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.630+0000 7fd01effd700 1 -- 172.21.15.19:0/852520728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02019fa20 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.631+0000 7fd01effd700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd02019f4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0100056d0 tx=0x7fd0100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.631+0000 7fd007fff700 1 -- 172.21.15.19:0/852520728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd010027070 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.631+0000 7fd007fff700 1 -- 172.21.15.19:0/852520728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01000be10 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.631+0000 7fd007fff700 1 -- 172.21.15.19:0/852520728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd010022d20 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.631+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02019fd10 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.631+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd020199840 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.632+0000 7fd007fff700 1 -- 172.21.15.19:0/852520728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fd01002a070 con 0x7fd020109a10 2024-04-25T10:46:04.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.632+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd020105120 con 0x7fd020109a10 2024-04-25T10:46:04.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.632+0000 7fd007fff700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd00803d670 0x7fd00803fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:04.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.632+0000 7fd007fff700 1 -- 172.21.15.19:0/852520728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd01001e030 con 0x7fd020109a10 2024-04-25T10:46:04.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.632+0000 7fd01e7fc700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd00803d670 0x7fd00803fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:04.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.633+0000 7fd01e7fc700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd00803d670 0x7fd00803fb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd014006fd0 tx=0x7fd014006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:04.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.638+0000 7fd007fff700 1 -- 172.21.15.19:0/852520728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd010013070 con 0x7fd020109a10 2024-04-25T10:46:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.851+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd02019abb0 con 0x7fd00803d670 2024-04-25T10:46:04.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.852+0000 7fd007fff700 1 -- 172.21.15.19:0/852520728 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fd02019abb0 con 0x7fd00803d670 2024-04-25T10:46:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:46:04.853 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd00803d670 msgr2=0x7fd00803fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd00803d670 0x7fd00803fb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd014006fd0 tx=0x7fd014006e40 comp rx=0 tx=0).stop 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 msgr2=0x7fd02019f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd02019f4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0100056d0 tx=0x7fd0100057b0 comp rx=0 tx=0).stop 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 shutdown_connections 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd00803d670 0x7fd00803fb80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 --2- 172.21.15.19:0/852520728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd020109a10 0x7fd02019f4e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 >> 172.21.15.19:0/852520728 conn(0x7fd0200ff560 msgr2=0x7fd0201004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 shutdown_connections 2024-04-25T10:46:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:04.854+0000 7fd0253ee700 1 -- 172.21.15.19:0/852520728 wait complete. 2024-04-25T10:46:05.461 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T10:46:05.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T10:46:05.706 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:46:05.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:05 smithi019 ceph-mon[29209]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:05.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:06.728 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:06 smithi019 ceph-mon[29209]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:46:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.213+0000 7f1848701700 1 -- 172.21.15.19:0/3316444580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 msgr2=0x7f1840108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.213+0000 7f1848701700 1 --2- 172.21.15.19:0/3316444580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f1840108d20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f183c009910 tx=0x7f183c009c20 comp rx=0 tx=0).stop 2024-04-25T10:46:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.213+0000 7f1848701700 1 -- 172.21.15.19:0/3316444580 shutdown_connections 2024-04-25T10:46:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.213+0000 7f1848701700 1 --2- 172.21.15.19:0/3316444580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f1840108d20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.213+0000 7f1848701700 1 -- 172.21.15.19:0/3316444580 >> 172.21.15.19:0/3316444580 conn(0x7f18400fdd00 msgr2=0x7f1840100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.213+0000 7f1848701700 1 -- 172.21.15.19:0/3316444580 shutdown_connections 2024-04-25T10:46:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.213+0000 7f1848701700 1 -- 172.21.15.19:0/3316444580 wait complete. 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.214+0000 7f1848701700 1 Processor -- start 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.214+0000 7f1848701700 1 -- start start 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.214+0000 7f1848701700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f184019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.214+0000 7f1848701700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f184010ce60 con 0x7f1840108910 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.214+0000 7f184649d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f184019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.214+0000 7f184649d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f184019f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41046/0 (socket says 172.21.15.19:41046) 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.214+0000 7f184649d700 1 -- 172.21.15.19:0/1888152915 learned_addr learned my addr 172.21.15.19:0/1888152915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.215+0000 7f184649d700 1 -- 172.21.15.19:0/1888152915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184019fa90 con 0x7f1840108910 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.215+0000 7f184649d700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f184019f550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f183c0056d0 tx=0x7f183c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.215+0000 7f18377fe700 1 -- 172.21.15.19:0/1888152915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f183c027070 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.215+0000 7f18377fe700 1 -- 172.21.15.19:0/1888152915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183c00be10 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.215+0000 7f18377fe700 1 -- 172.21.15.19:0/1888152915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f183c022d20 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.215+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184019fd80 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.215+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18401998b0 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.216+0000 7f18377fe700 1 -- 172.21.15.19:0/1888152915 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f183c02a070 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.216+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1840104160 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.216+0000 7f18377fe700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f182c03d670 0x7f182c03fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.216+0000 7f18377fe700 1 -- 172.21.15.19:0/1888152915 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f183c01e030 con 0x7f1840108910 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.216+0000 7f1845c9c700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f182c03d670 0x7f182c03fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.217+0000 7f1845c9c700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f182c03d670 0x7f182c03fb80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1830006fd0 tx=0x7f1830006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.222+0000 7f18377fe700 1 -- 172.21.15.19:0/1888152915 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f183c013070 con 0x7f1840108910 2024-04-25T10:46:07.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.438+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f184019ffb0 con 0x7f1840108910 2024-04-25T10:46:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.626+0000 7f18377fe700 1 -- 172.21.15.19:0/1888152915 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f183c02b3b0 con 0x7f1840108910 2024-04-25T10:46:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f182c03d670 msgr2=0x7f182c03fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f182c03d670 0x7f182c03fb80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1830006fd0 tx=0x7f1830006e40 comp rx=0 tx=0).stop 2024-04-25T10:46:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 msgr2=0x7f184019f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f184019f550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f183c0056d0 tx=0x7f183c0057b0 comp rx=0 tx=0).stop 2024-04-25T10:46:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 shutdown_connections 2024-04-25T10:46:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f182c03d670 0x7f182c03fb80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 --2- 172.21.15.19:0/1888152915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1840108910 0x7f184019f550 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.628+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 >> 172.21.15.19:0/1888152915 conn(0x7f18400fdd00 msgr2=0x7f18401063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.629+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 shutdown_connections 2024-04-25T10:46:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:07.629+0000 7f1848701700 1 -- 172.21.15.19:0/1888152915 wait complete. 2024-04-25T10:46:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-04-25T10:46:07.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:07 smithi019 ceph-mon[29209]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:07.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:07 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1888152915' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T10:46:08.278 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-04-25T10:46:08.278 INFO:tasks.cephadm:Adding mon.b on smithi112 2024-04-25T10:46:08.278 INFO:tasks.cephadm:Adding mon.c on smithi169 2024-04-25T10:46:08.278 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi112:172.21.15.112=b;smithi169:172.21.15.169=c' 2024-04-25T10:46:08.519 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1888152915' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:09.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.986+0000 7f3413598700 1 -- 172.21.15.169:0/105282578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 msgr2=0x7f340c107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.986+0000 7f3413598700 1 --2- 172.21.15.169:0/105282578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c107d30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3408009910 tx=0x7f3408009c20 comp rx=0 tx=0).stop 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.986+0000 7f3413598700 1 -- 172.21.15.169:0/105282578 shutdown_connections 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.986+0000 7f3413598700 1 --2- 172.21.15.169:0/105282578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c107d30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.986+0000 7f3413598700 1 -- 172.21.15.169:0/105282578 >> 172.21.15.169:0/105282578 conn(0x7f340c078c50 msgr2=0x7f340c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.986+0000 7f3413598700 1 -- 172.21.15.169:0/105282578 shutdown_connections 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.986+0000 7f3413598700 1 -- 172.21.15.169:0/105282578 wait complete. 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.987+0000 7f3413598700 1 Processor -- start 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.987+0000 7f3413598700 1 -- start start 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.988+0000 7f3413598700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:09.991 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.988+0000 7f3413598700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10cfc0 con 0x7f340c107920 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.988+0000 7f3411334700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.988+0000 7f3411334700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.169:37188/0 (socket says 172.21.15.169:37188) 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.988+0000 7f3411334700 1 -- 172.21.15.169:0/3254744258 learned_addr learned my addr 172.21.15.169:0/3254744258 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.988+0000 7f3411334700 1 -- 172.21.15.169:0/3254744258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c102ea0 con 0x7f340c107920 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.988+0000 7f3411334700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c19a6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34080056d0 tx=0x7f34080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.989+0000 7f34027fc700 1 -- 172.21.15.169:0/3254744258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3408026070 con 0x7f340c107920 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.989+0000 7f34027fc700 1 -- 172.21.15.169:0/3254744258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340800bc00 con 0x7f340c107920 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.989+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c19ac70 con 0x7f340c107920 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.989+0000 7f34027fc700 1 -- 172.21.15.169:0/3254744258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3408003920 con 0x7f340c107920 2024-04-25T10:46:09.992 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.989+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c19b130 con 0x7f340c107920 2024-04-25T10:46:09.993 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.990+0000 7f34027fc700 1 -- 172.21.15.169:0/3254744258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f3408029070 con 0x7f340c107920 2024-04-25T10:46:09.993 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.990+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c1040b0 con 0x7f340c107920 2024-04-25T10:46:09.993 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.990+0000 7f34027fc700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f33f803d610 0x7f33f803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:09.993 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.990+0000 7f34027fc700 1 -- 172.21.15.169:0/3254744258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f340801e070 con 0x7f340c107920 2024-04-25T10:46:09.993 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.990+0000 7f3410b33700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f33f803d610 0x7f33f803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:09.993 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.991+0000 7f3410b33700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f33f803d610 0x7f33f803fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006fd0 tx=0x7f33fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:09.998 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:09.996+0000 7f34027fc700 1 -- 172.21.15.169:0/3254744258 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3408013070 con 0x7f340c107920 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:09 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:10.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.204+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi112:172.21.15.112=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}) -- 0x7f340c1025f0 con 0x7f33f803d610 2024-04-25T10:46:10.214 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.212+0000 7f34027fc700 1 -- 172.21.15.169:0/3254744258 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f340c1025f0 con 0x7f33f803d610 2024-04-25T10:46:10.214 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-04-25T10:46:10.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f33f803d610 msgr2=0x7f33f803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:10.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f33f803d610 0x7f33f803fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006fd0 tx=0x7f33fc006e40 comp rx=0 tx=0).stop 2024-04-25T10:46:10.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 msgr2=0x7f340c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:10.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c19a6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34080056d0 tx=0x7f34080057b0 comp rx=0 tx=0).stop 2024-04-25T10:46:10.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 shutdown_connections 2024-04-25T10:46:10.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f33f803d610 0x7f33f803fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:10.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 --2- 172.21.15.169:0/3254744258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c107920 0x7f340c19a6d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:10.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 >> 172.21.15.169:0/3254744258 conn(0x7f340c078c50 msgr2=0x7f340c109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:10.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 shutdown_connections 2024-04-25T10:46:10.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:10.214+0000 7f3413598700 1 -- 172.21.15.169:0/3254744258 wait complete. 2024-04-25T10:46:10.820 DEBUG:teuthology.orchestra.run.smithi112:mon.b> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T10:46:10.822 DEBUG:teuthology.orchestra.run.smithi169:mon.c> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T10:46:10.824 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T10:46:10.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi112:172.21.15.112=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi112:172.21.15.112=b;smithi169:172.21.15.169=c;count:3 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:11.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:11.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:11.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:11 smithi019 ceph-mon[29209]: Deploying daemon mon.c on smithi169 2024-04-25T10:46:12.431 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:13.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:13 smithi019 ceph-mon[29209]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:14.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:13 smithi169 systemd[1]: Starting Ceph mon.c for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:46:14.569 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.566+0000 7fae8e06a700 1 -- 172.21.15.169:0/1411219543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 msgr2=0x7fae88108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:14.569 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.566+0000 7fae8e06a700 1 --2- 172.21.15.169:0/1411219543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88108d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fae78009910 tx=0x7fae78009c20 comp rx=0 tx=0).stop 2024-04-25T10:46:14.569 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.566+0000 7fae8e06a700 1 -- 172.21.15.169:0/1411219543 shutdown_connections 2024-04-25T10:46:14.569 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.566+0000 7fae8e06a700 1 --2- 172.21.15.169:0/1411219543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88108d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:14.569 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.566+0000 7fae8e06a700 1 -- 172.21.15.169:0/1411219543 >> 172.21.15.169:0/1411219543 conn(0x7fae880fe490 msgr2=0x7fae881008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:14.569 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.566+0000 7fae8e06a700 1 -- 172.21.15.169:0/1411219543 shutdown_connections 2024-04-25T10:46:14.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.566+0000 7fae8e06a700 1 -- 172.21.15.169:0/1411219543 wait complete. 2024-04-25T10:46:14.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.567+0000 7fae8e06a700 1 Processor -- start 2024-04-25T10:46:14.570 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 podman[33007]: 2024-04-25 10:46:14.346431819 +0000 UTC m=+0.019690042 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:14.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.567+0000 7fae8e06a700 1 -- start start 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.567+0000 7fae8e06a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.567+0000 7fae8e06a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810ce90 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.567+0000 7fae877fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae877fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.169:37196/0 (socket says 172.21.15.169:37196) 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae877fe700 1 -- 172.21.15.169:0/1976521040 learned_addr learned my addr 172.21.15.169:0/1976521040 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae877fe700 1 -- 172.21.15.169:0/1976521040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8806c790 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae877fe700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88198400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae780056d0 tx=0x7fae780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae857fa700 1 -- 172.21.15.169:0/1976521040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae78027070 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae857fa700 1 -- 172.21.15.169:0/1976521040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7800be10 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae88198940 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae857fa700 1 -- 172.21.15.169:0/1976521040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae78022d20 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.568+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae88198e00 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.569+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae88104050 con 0x7fae88108940 2024-04-25T10:46:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.569+0000 7fae857fa700 1 -- 172.21.15.169:0/1976521040 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fae7802a070 con 0x7fae88108940 2024-04-25T10:46:14.573 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.569+0000 7fae857fa700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fae6803d610 0x7fae6803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:14.573 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.570+0000 7fae857fa700 1 -- 172.21.15.169:0/1976521040 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fae7801e030 con 0x7fae88108940 2024-04-25T10:46:14.573 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.570+0000 7fae7effd700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fae6803d610 0x7fae6803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:14.573 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.570+0000 7fae7effd700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fae6803d610 0x7fae6803fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fae70006fd0 tx=0x7fae70006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:14.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.575+0000 7fae857fa700 1 -- 172.21.15.169:0/1976521040 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae78013070 con 0x7fae88108940 2024-04-25T10:46:14.858 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 podman[33007]: 2024-04-25 10:46:14.688636946 +0000 UTC m=+0.361895159 container create 9313eeb8a1be24f016b0ef40156fef5e3de7d66b0d628053c011a8c4f63cffb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:46:14.858 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 podman[33007]: 2024-04-25 10:46:14.857694688 +0000 UTC m=+0.530952902 container init 9313eeb8a1be24f016b0ef40156fef5e3de7d66b0d628053c011a8c4f63cffb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:46:14.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.863+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fae8806c020 con 0x7fae88108940 2024-04-25T10:46:14.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.864+0000 7fae857fa700 1 -- 172.21.15.169:0/1976521040 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fae7802b3b0 con 0x7fae88108940 2024-04-25T10:46:14.866 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:46:14.866 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","modified":"2024-04-25T10:44:24.525437Z","created":"2024-04-25T10:44:24.525437Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T10:46:14.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fae6803d610 msgr2=0x7fae6803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:14.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fae6803d610 0x7fae6803fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fae70006fd0 tx=0x7fae70006e40 comp rx=0 tx=0).stop 2024-04-25T10:46:14.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 msgr2=0x7fae88198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:14.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88198400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae780056d0 tx=0x7fae780057b0 comp rx=0 tx=0).stop 2024-04-25T10:46:14.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 shutdown_connections 2024-04-25T10:46:14.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fae6803d610 0x7fae6803fb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:14.869 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 --2- 172.21.15.169:0/1976521040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae88108940 0x7fae88198400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:14.869 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 >> 172.21.15.169:0/1976521040 conn(0x7fae880fe490 msgr2=0x7fae880ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:14.869 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 shutdown_connections 2024-04-25T10:46:14.869 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:14.866+0000 7fae8e06a700 1 -- 172.21.15.169:0/1976521040 wait complete. 2024-04-25T10:46:14.869 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-04-25T10:46:15.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 podman[33007]: 2024-04-25 10:46:14.86129844 +0000 UTC m=+0.534556656 container start 9313eeb8a1be24f016b0ef40156fef5e3de7d66b0d628053c011a8c4f63cffb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T10:46:15.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pidfile_write: ignore empty --pid-file 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: load: jerasure load: lrc 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: RocksDB version: 7.9.2 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Git sha 0 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: DB SUMMARY 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: DB Session ID: 2MP8JZEI3S071VZVWQT6 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: CURRENT file: CURRENT 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T10:46:15.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.error_if_exists: 0 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.create_if_missing: 0 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.paranoid_checks: 1 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.env: 0x558e8c2a7500 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.info_log: 0x558e8deddfc0 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.statistics: (nil) 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.use_fsync: 0 2024-04-25T10:46:15.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_log_file_size: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.allow_fallocate: 1 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.use_direct_reads: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.db_log_dir: 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.wal_dir: 2024-04-25T10:46:15.255 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.write_buffer_manager: 0x558e8deae8c0 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T10:46:15.256 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.unordered_write: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.row_cache: None 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.wal_filter: None 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.two_write_queues: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.wal_compression: 0 2024-04-25T10:46:15.257 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.atomic_flush: 0 2024-04-25T10:46:15.258 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T10:46:15.258 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T10:46:15.259 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T10:46:15.259 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.log_readahead_size: 0 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_background_jobs: 2 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_background_compactions: -1 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_subcompactions: 1 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T10:46:15.260 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_open_files: -1 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_background_flushes: -1 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Compression algorithms supported: 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T10:46:15.261 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kZSTD supported: 0 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kXpressCompression supported: 0 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kZlibCompression supported: 1 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kSnappyCompression supported: 1 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kLZ4Compression supported: 1 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: kBZip2Compression supported: 0 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.merge_operator: 2024-04-25T10:46:15.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_filter: None 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558e8dfe41a0) 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: index_type: 0 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: data_block_index_type: 0 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: index_shortening: 1 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: checksum: 4 2024-04-25T10:46:15.263 INFO:journalctl@ceph.mon.c.smithi169.stdout: no_block_cache: 0 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache: 0x558e8dee11f0 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache_options: 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: capacity : 536870912 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: num_shard_bits : 4 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: strict_capacity_limit : 0 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache_compressed: (nil) 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: persistent_cache: (nil) 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_size: 4096 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_size_deviation: 10 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_restart_interval: 16 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: index_block_restart_interval: 1 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: metadata_block_size: 4096 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: partition_filters: 0 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: use_delta_encoding: 1 2024-04-25T10:46:15.264 INFO:journalctl@ceph.mon.c.smithi169.stdout: filter_policy: bloomfilter 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: whole_key_filtering: 1 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: verify_compression: 0 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: format_version: 5 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: enable_index_compression: 1 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_align: 0 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: max_auto_readahead_size: 262144 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: prepopulate_block_cache: 0 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: initial_auto_readahead_size: 8192 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression: NoCompression 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T10:46:15.265 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.num_levels: 7 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:46:15.266 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T10:46:15.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T10:46:15.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.inplace_update_support: 0 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T10:46:15.269 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.bloom_locality: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.max_successive_merges: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.ttl: 2592000 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.enable_blob_files: false 2024-04-25T10:46:15.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.min_blob_size: 0 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 075d9d4b-e762-42f9-b070-f6068845f883 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714041974903669, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T10:46:15.271 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714041974904107, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714041974, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "075d9d4b-e762-42f9-b070-f6068845f883", "db_session_id": "2MP8JZEI3S071VZVWQT6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714041974904188, "job": 1, "event": "recovery_finished"} 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558e8dec5c00 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: DB pointer 0x558e92cb8000 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** DB Stats ** 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:46:15.272 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** Compaction Stats [default] ** 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** Compaction Stats [default] ** 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T10:46:15.273 INFO:journalctl@ceph.mon.c.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T10:46:15.274 INFO:journalctl@ceph.mon.c.smithi169.stdout: Block cache BinnedLRUCache@0x558e8dee11f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: using public_addr v2:172.21.15.169:0/0 -> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: starting mon.c rank -1 at public addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] at bind addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(???) e0 preinit fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: e1 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: legacy client fscid: -1 2024-04-25T10:46:15.275 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout: No filesystems configured 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:15.276 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Deploying cephadm binary to smithi169 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:46:15.277 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Added host smithi169 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1888152915' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T10:46:15.278 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1888152915' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.279 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi112:172.21.15.112=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi112:172.21.15.112=b;smithi169:172.21.15.169=c;count:3 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: Deploying daemon mon.c on smithi169 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:15.280 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 ceph-mon[33049]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T10:46:15.281 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 bash[33007]: 9313eeb8a1be24f016b0ef40156fef5e3de7d66b0d628053c011a8c4f63cffb3 2024-04-25T10:46:15.281 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:14 smithi169 systemd[1]: Started Ceph mon.c for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:46:16.564 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T10:46:16.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T10:46:16.810 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:46:20.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: mon.a calling monitor election 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: mon.c calling monitor election 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: monmap epoch 2 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: last_changed 2024-04-25T10:46:14.928942+0000 2024-04-25T10:46:20.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: min_mon_release 19 (squid) 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: election_strategy: 1 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: fsmap 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: mgrmap e13: x(active, since 50s) 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: overall HEALTH_OK 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:20.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:20.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:46:20.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: mon.a calling monitor election 2024-04-25T10:46:20.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:20.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: mon.c calling monitor election 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: monmap epoch 2 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: last_changed 2024-04-25T10:46:14.928942+0000 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: min_mon_release 19 (squid) 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: election_strategy: 1 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: fsmap 2024-04-25T10:46:20.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: mgrmap e13: x(active, since 50s) 2024-04-25T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: overall HEALTH_OK 2024-04-25T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:21 smithi169 ceph-mon[33049]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:21 smithi169 ceph-mon[33049]: Deploying daemon mon.b on smithi112 2024-04-25T10:46:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.358+0000 7f8dca24b700 1 -- 172.21.15.169:0/3534860191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac0035c0 msgr2=0x7f8dac005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.358+0000 7f8dca24b700 1 --2- 172.21.15.169:0/3534860191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac0035c0 0x7f8dac005aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4103a40 tx=0x7f8db800feb0 comp rx=0 tx=0).stop 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.358+0000 7f8dca24b700 1 -- 172.21.15.169:0/3534860191 shutdown_connections 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.358+0000 7f8dca24b700 1 --2- 172.21.15.169:0/3534860191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac0035c0 0x7f8dac005aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.358+0000 7f8dca24b700 1 -- 172.21.15.169:0/3534860191 >> 172.21.15.169:0/3534860191 conn(0x7f8dc40ff670 msgr2=0x7f8dc4101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.358+0000 7f8dca24b700 1 -- 172.21.15.169:0/3534860191 shutdown_connections 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.358+0000 7f8dca24b700 1 -- 172.21.15.169:0/3534860191 wait complete. 2024-04-25T10:46:21.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.359+0000 7f8dca24b700 1 Processor -- start 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.359+0000 7f8dca24b700 1 -- start start 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.359+0000 7f8dca24b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 0x7f8dc41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.359+0000 7f8dca24b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc41a1c50 0x7f8dc419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.359+0000 7f8dca24b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410bb10 con 0x7f8dc41a1c50 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dca24b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410b990 con 0x7f8dac0035c0 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 0x7f8dc41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 0x7f8dc41a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54144/0 (socket says 172.21.15.169:54144) 2024-04-25T10:46:21.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc37fe700 1 -- 172.21.15.169:0/2578852695 learned_addr learned my addr 172.21.15.169:0/2578852695 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc2ffd700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc41a1c50 0x7f8dc419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc37fe700 1 -- 172.21.15.169:0/2578852695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 msgr2=0x7f8dc41a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc37fe700 1 -- 172.21.15.169:0/2578852695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 msgr2=0x7f8dc41a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc37fe700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 0x7f8dc41a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc37fe700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 0x7f8dc41a1710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc2ffd700 1 -- 172.21.15.169:0/2578852695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 msgr2=0x7f8dc41a1710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc2ffd700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 0x7f8dc41a1710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc2ffd700 1 -- 172.21.15.169:0/2578852695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc41a2220 con 0x7f8dc41a1c50 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.360+0000 7f8dc2ffd700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc41a1c50 0x7f8dc419b930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8db400b4d0 tx=0x7f8db400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.361+0000 7f8dc0ff9700 1 -- 172.21.15.169:0/2578852695 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8db4011630 con 0x7f8dc41a1c50 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.361+0000 7f8dc0ff9700 1 -- 172.21.15.169:0/2578852695 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db4011c80 con 0x7f8dc41a1c50 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.361+0000 7f8dc0ff9700 1 -- 172.21.15.169:0/2578852695 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8db4010da0 con 0x7f8dc41a1c50 2024-04-25T10:46:21.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.361+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc419bed0 con 0x7f8dc41a1c50 2024-04-25T10:46:21.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.361+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc419c4e0 con 0x7f8dc41a1c50 2024-04-25T10:46:21.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:21 smithi019 ceph-mon[29209]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:21.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:21 smithi019 ceph-mon[29209]: Deploying daemon mon.b on smithi112 2024-04-25T10:46:21.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:21.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.362+0000 7f8dc0ff9700 1 -- 172.21.15.169:0/2578852695 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f8db4018020 con 0x7f8dc41a1c50 2024-04-25T10:46:21.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.362+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc41051e0 con 0x7f8dc41a1c50 2024-04-25T10:46:21.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.362+0000 7f8dc0ff9700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8db003d400 0x7f8db003f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:21.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.362+0000 7f8dc0ff9700 1 -- 172.21.15.169:0/2578852695 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8db4050fe0 con 0x7f8dc41a1c50 2024-04-25T10:46:21.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.362+0000 7f8dc37fe700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8db003d400 0x7f8db003f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:21.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.363+0000 7f8dc37fe700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8db003d400 0x7f8db003f910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8db800fa60 tx=0x7f8db800f4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:21.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.368+0000 7f8dc0ff9700 1 -- 172.21.15.169:0/2578852695 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8db401f020 con 0x7f8dc41a1c50 2024-04-25T10:46:21.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.652+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8dc419ce60 con 0x7f8dc41a1c50 2024-04-25T10:46:21.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.653+0000 7f8dc0ff9700 1 -- 172.21.15.169:0/2578852695 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8db40187b0 con 0x7f8dc41a1c50 2024-04-25T10:46:21.654 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:46:21.654 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","modified":"2024-04-25T10:46:14.928942Z","created":"2024-04-25T10:44:24.525437Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T10:46:21.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8db003d400 msgr2=0x7f8db003f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:21.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8db003d400 0x7f8db003f910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8db800fa60 tx=0x7f8db800f4a0 comp rx=0 tx=0).stop 2024-04-25T10:46:21.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc41a1c50 msgr2=0x7f8dc419b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc41a1c50 0x7f8dc419b930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8db400b4d0 tx=0x7f8db400b890 comp rx=0 tx=0).stop 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 shutdown_connections 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8db003d400 0x7f8db003f910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0035c0 0x7f8dc41a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 --2- 172.21.15.169:0/2578852695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc41a1c50 0x7f8dc419b930 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 >> 172.21.15.169:0/2578852695 conn(0x7f8dc40ff670 msgr2=0x7f8dc41022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 shutdown_connections 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:21.655+0000 7f8dca24b700 1 -- 172.21.15.169:0/2578852695 wait complete. 2024-04-25T10:46:21.657 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-04-25T10:46:22.335 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:22 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/2578852695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:46:22.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:22 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/2578852695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:46:23.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:23 smithi019 ceph-mon[29209]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:23.364 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T10:46:23.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T10:46:23.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:23 smithi169 ceph-mon[33049]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:46:23.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:23 smithi112 ceph-mon[32799]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T10:46:29.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: mon.c calling monitor election 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: mon.a calling monitor election 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:46:29.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: monmap epoch 3 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: last_changed 2024-04-25T10:46:23.625813+0000 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: min_mon_release 19 (squid) 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: election_strategy: 1 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:29.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T10:46:29.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: fsmap 2024-04-25T10:46:29.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:29.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: mgrmap e13: x(active, since 58s) 2024-04-25T10:46:29.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: overall HEALTH_OK 2024-04-25T10:46:29.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:29.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:29.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:46:29.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: mon.c calling monitor election 2024-04-25T10:46:29.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: mon.a calling monitor election 2024-04-25T10:46:29.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: monmap epoch 3 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: last_changed 2024-04-25T10:46:23.625813+0000 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: min_mon_release 19 (squid) 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: election_strategy: 1 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:29.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T10:46:29.115 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: fsmap 2024-04-25T10:46:29.115 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:29.115 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: mgrmap e13: x(active, since 58s) 2024-04-25T10:46:29.115 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: overall HEALTH_OK 2024-04-25T10:46:29.115 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:29.115 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:29.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.335+0000 7f5ec55b2700 1 -- 172.21.15.169:0/2511479847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 msgr2=0x7f5eac005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:29.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.335+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/2511479847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 0x7f5eac005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005fd0 tx=0x7f5eb000ff70 comp rx=0 tx=0).stop 2024-04-25T10:46:29.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.335+0000 7f5ec55b2700 1 -- 172.21.15.169:0/2511479847 shutdown_connections 2024-04-25T10:46:29.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.335+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/2511479847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 0x7f5eac005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:29.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.335+0000 7f5ec55b2700 1 -- 172.21.15.169:0/2511479847 >> 172.21.15.169:0/2511479847 conn(0x7f5ec00ff620 msgr2=0x7f5ec0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:29.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.335+0000 7f5ec55b2700 1 -- 172.21.15.169:0/2511479847 shutdown_connections 2024-04-25T10:46:29.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.335+0000 7f5ec55b2700 1 -- 172.21.15.169:0/2511479847 wait complete. 2024-04-25T10:46:29.341 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 Processor -- start 2024-04-25T10:46:29.341 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 -- start start 2024-04-25T10:46:29.341 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 0x7f5ec0075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 0x7f5ec00762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 0x7f5ec01a8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec010ba90 con 0x7f5ec0109ba0 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec010b910 con 0x7f5ec01a5c10 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.336+0000 7f5ec55b2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec010bc10 con 0x7f5ec0075ed0 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5eb67fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 0x7f5ec00762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5eb67fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 0x7f5ec00762e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56962/0 (socket says 172.21.15.169:56962) 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5eb67fc700 1 -- 172.21.15.169:0/812289196 learned_addr learned my addr 172.21.15.169:0/812289196 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:46:29.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5eb67fc700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 msgr2=0x7f5ec00762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T10:46:29.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5eb67fc700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 msgr2=0x7f5ec00762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:46:29.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5eb67fc700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 0x7f5ec00762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:46:29.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebf7fe700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 0x7f5ec01a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:29.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5eb67fc700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 0x7f5ec00762e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:46:29.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebeffd700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 0x7f5ec0075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:29.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebf7fe700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 msgr2=0x7f5ec01a8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T10:46:29.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebf7fe700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 msgr2=0x7f5ec01a8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebf7fe700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 0x7f5ec01a8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebf7fe700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 0x7f5ec01a8030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebeffd700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 msgr2=0x7f5ec00762e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebeffd700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 0x7f5ec00762e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebeffd700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 msgr2=0x7f5ec01a8030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebeffd700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 0x7f5ec01a8030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.337+0000 7f5ebeffd700 1 -- 172.21.15.169:0/812289196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec01a86b0 con 0x7f5ec0109ba0 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.338+0000 7f5ebeffd700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 0x7f5ec0075990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.338+0000 7f5ebcff9700 1 -- 172.21.15.169:0/812289196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0022040 con 0x7f5ec0109ba0 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.338+0000 7f5ebcff9700 1 -- 172.21.15.169:0/812289196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb0004220 con 0x7f5ec0109ba0 2024-04-25T10:46:29.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.338+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec01a8940 con 0x7f5ec0109ba0 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.338+0000 7f5ebcff9700 1 -- 172.21.15.169:0/812289196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0004690 con 0x7f5ec0109ba0 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.338+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec01a8e80 con 0x7f5ec0109ba0 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.339+0000 7f5ebcff9700 1 -- 172.21.15.169:0/812289196 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f5eb0004850 con 0x7f5ec0109ba0 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.339+0000 7f5ebcff9700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5ea003daa0 0x7f5ea003ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.339+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec0064cf0 con 0x7f5ec0109ba0 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.339+0000 7f5ebcff9700 1 -- 172.21.15.169:0/812289196 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5eb0054540 con 0x7f5ec0109ba0 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.339+0000 7f5eb67fc700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5ea003daa0 0x7f5ea003ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:29.345 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.340+0000 7f5eb67fc700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5ea003daa0 0x7f5ea003ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0102b30 tx=0x7f5ea8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:29.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.345+0000 7f5ebcff9700 1 -- 172.21.15.169:0/812289196 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5eb0010870 con 0x7f5ec0109ba0 2024-04-25T10:46:29.670 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.669+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5ec002e510 con 0x7f5ec0109ba0 2024-04-25T10:46:29.671 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.669+0000 7f5ebcff9700 1 -- 172.21.15.169:0/812289196 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f5eb001b300 con 0x7f5ec0109ba0 2024-04-25T10:46:29.671 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:46:29.671 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":3,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","modified":"2024-04-25T10:46:23.625813Z","created":"2024-04-25T10:44:24.525437Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T10:46:29.673 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5ea003daa0 msgr2=0x7f5ea003ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:29.673 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5ea003daa0 0x7f5ea003ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0102b30 tx=0x7f5ea8008040 comp rx=0 tx=0).stop 2024-04-25T10:46:29.673 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 msgr2=0x7f5ec0075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:29.673 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 0x7f5ec0075990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb0003d80 comp rx=0 tx=0).stop 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 shutdown_connections 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5ea003daa0 0x7f5ea003ffb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec01a5c10 0x7f5ec01a8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec0109ba0 0x7f5ec0075990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 --2- 172.21.15.169:0/812289196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0075ed0 0x7f5ec00762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.672+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 >> 172.21.15.169:0/812289196 conn(0x7f5ec00ff620 msgr2=0x7f5ec0107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.673+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 shutdown_connections 2024-04-25T10:46:29.674 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:29.673+0000 7f5ec55b2700 1 -- 172.21.15.169:0/812289196 wait complete. 2024-04-25T10:46:29.675 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 3 2024-04-25T10:46:29.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:29 smithi169 ceph-mon[33049]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:29.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:29.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:29.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:29.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:29.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/812289196' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:46:30.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:29 smithi019 ceph-mon[29209]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:30.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:30.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:30.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:30.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:30.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/812289196' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:46:30.261 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T10:46:30.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T10:46:30.518 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: mon.c calling monitor election 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: mon.a calling monitor election 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: monmap epoch 3 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: last_changed 2024-04-25T10:46:23.625813+0000 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: min_mon_release 19 (squid) 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: election_strategy: 1 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T10:46:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: fsmap 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: mgrmap e13: x(active, since 58s) 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: overall HEALTH_OK 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:30 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/812289196' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:46:31.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: mon.b calling monitor election 2024-04-25T10:46:31.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:31.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: mon.b calling monitor election 2024-04-25T10:46:31.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: mon.a calling monitor election 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: mon.c calling monitor election 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: monmap epoch 3 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: last_changed 2024-04-25T10:46:23.625813+0000 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: min_mon_release 19 (squid) 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: election_strategy: 1 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: fsmap 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: mgrmap e13: x(active, since 60s) 2024-04-25T10:46:31.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: overall HEALTH_OK 2024-04-25T10:46:31.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:31.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:31.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T10:46:31.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:46:31.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:31 smithi019 ceph-mon[29209]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:46:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: mon.b calling monitor election 2024-04-25T10:46:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: mon.b calling monitor election 2024-04-25T10:46:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: mon.a calling monitor election 2024-04-25T10:46:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: mon.c calling monitor election 2024-04-25T10:46:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T10:46:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: monmap epoch 3 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: last_changed 2024-04-25T10:46:23.625813+0000 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: min_mon_release 19 (squid) 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: election_strategy: 1 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: fsmap 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: mgrmap e13: x(active, since 60s) 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: overall HEALTH_OK 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:31.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:31.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T10:46:31.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:46:31.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:31 smithi169 ceph-mon[33049]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: mon.b calling monitor election 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: mon.b calling monitor election 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: mon.a calling monitor election 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: mon.c calling monitor election 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: monmap epoch 3 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: last_changed 2024-04-25T10:46:23.625813+0000 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: created 2024-04-25T10:44:24.525437+0000 2024-04-25T10:46:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: min_mon_release 19 (squid) 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: election_strategy: 1 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: fsmap 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: mgrmap e13: x(active, since 60s) 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: overall HEALTH_OK 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:46:31.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:31 smithi112 ceph-mon[32799]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:46:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.176+0000 7fb23e71d700 1 -- 172.21.15.19:0/787002819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 msgr2=0x7fb238109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.176+0000 7fb23e71d700 1 --2- 172.21.15.19:0/787002819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb238109e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb228009910 tx=0x7fb228009c20 comp rx=0 tx=0).stop 2024-04-25T10:46:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.176+0000 7fb23e71d700 1 -- 172.21.15.19:0/787002819 shutdown_connections 2024-04-25T10:46:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.176+0000 7fb23e71d700 1 --2- 172.21.15.19:0/787002819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb238109e20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.176+0000 7fb23e71d700 1 -- 172.21.15.19:0/787002819 >> 172.21.15.19:0/787002819 conn(0x7fb2380ff560 msgr2=0x7fb2381019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.176+0000 7fb23e71d700 1 -- 172.21.15.19:0/787002819 shutdown_connections 2024-04-25T10:46:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.176+0000 7fb23e71d700 1 -- 172.21.15.19:0/787002819 wait complete. 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.177+0000 7fb23e71d700 1 Processor -- start 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.177+0000 7fb23e71d700 1 -- start start 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb23e71d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb23819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb23e71d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23819fc60 0x7fb238199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb23e71d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238199e80 0x7fb23819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb23e71d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810df60 con 0x7fb238109a10 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb23e71d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810dde0 con 0x7fb23819fc60 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb23e71d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810e0e0 con 0x7fb238199e80 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb23819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb23819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56424/0 (socket says 172.21.15.19:56424) 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 -- 172.21.15.19:0/2282434093 learned_addr learned my addr 172.21.15.19:0/2282434093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:46:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb2377fe700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23819fc60 0x7fb238199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb23ccba700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238199e80 0x7fb23819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 -- 172.21.15.19:0/2282434093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238199e80 msgr2=0x7fb23819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238199e80 0x7fb23819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 -- 172.21.15.19:0/2282434093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23819fc60 msgr2=0x7fb238199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23819fc60 0x7fb238199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb237fff700 1 -- 172.21.15.19:0/2282434093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2381a65b0 con 0x7fb238109a10 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.178+0000 7fb2377fe700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23819fc60 0x7fb238199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:46:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.179+0000 7fb237fff700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb23819f720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb228004fb0 tx=0x7fb228005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:32.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.179+0000 7fb2357fa700 1 -- 172.21.15.19:0/2282434093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228004240 con 0x7fb238109a10 2024-04-25T10:46:32.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.179+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2381a6840 con 0x7fb238109a10 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.179+0000 7fb2357fa700 1 -- 172.21.15.19:0/2282434093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2280043e0 con 0x7fb238109a10 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.179+0000 7fb2357fa700 1 -- 172.21.15.19:0/2282434093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228003a00 con 0x7fb238109a10 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.179+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2381a6da0 con 0x7fb238109a10 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.180+0000 7fb2357fa700 1 -- 172.21.15.19:0/2282434093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb228003bc0 con 0x7fb238109a10 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.180+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb238064cf0 con 0x7fb238109a10 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.180+0000 7fb2357fa700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb22003daa0 0x7fb22003ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.180+0000 7fb2357fa700 1 -- 172.21.15.19:0/2282434093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb228052720 con 0x7fb238109a10 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.180+0000 7fb2377fe700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb22003daa0 0x7fb22003ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.181+0000 7fb2377fe700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb22003daa0 0x7fb22003ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2380ff9b0 tx=0x7fb22c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.185+0000 7fb2357fa700 1 -- 172.21.15.19:0/2282434093 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb228013070 con 0x7fb238109a10 2024-04-25T10:46:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.397+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb2381a7000 con 0x7fb238109a10 2024-04-25T10:46:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.398+0000 7fb2357fa700 1 -- 172.21.15.19:0/2282434093 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fb228020c30 con 0x7fb238109a10 2024-04-25T10:46:32.399 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:32.399 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T10:46:32.400 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:46:32.400 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-04-25T10:46:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb22003daa0 msgr2=0x7fb22003ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb22003daa0 0x7fb22003ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2380ff9b0 tx=0x7fb22c008040 comp rx=0 tx=0).stop 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 msgr2=0x7fb23819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb23819f720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb228004fb0 tx=0x7fb228005090 comp rx=0 tx=0).stop 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 shutdown_connections 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb22003daa0 0x7fb22003ffb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238199e80 0x7fb23819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb238109a10 0x7fb23819f720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 --2- 172.21.15.19:0/2282434093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23819fc60 0x7fb238199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 >> 172.21.15.19:0/2282434093 conn(0x7fb2380ff560 msgr2=0x7fb238100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 shutdown_connections 2024-04-25T10:46:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:32.400+0000 7fb23e71d700 1 -- 172.21.15.19:0/2282434093 wait complete. 2024-04-25T10:46:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T10:46:32.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:32.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:32.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:32.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: Reconfiguring daemon mon.a on smithi019 2024-04-25T10:46:32.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: Reconfiguring daemon mon.a on smithi019 2024-04-25T10:46:32.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: Reconfiguring daemon mon.a on smithi019 2024-04-25T10:46:32.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:46:33.087 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T10:46:33.088 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:46:33.088 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T10:46:33.115 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:46:33.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:46:33.181 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:46:33.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T10:46:33.213 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:46:33.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:46:33.281 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:46:33.281 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T10:46:33.312 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:46:33.313 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:46:33.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:33 smithi019 ceph-mon[29209]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:33.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2282434093' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:33.381 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-04-25T10:46:33.381 INFO:tasks.cephadm:Adding mgr.y on smithi112 2024-04-25T10:46:33.381 INFO:tasks.cephadm:Adding mgr.z on smithi169 2024-04-25T10:46:33.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi112=y;smithi169=z' 2024-04-25T10:46:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:33 smithi112 ceph-mon[32799]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:33.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2282434093' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:33.654 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:46:33.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:33 smithi169 ceph-mon[33049]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:33.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2282434093' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:34.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.973+0000 7f79d25da700 1 -- 172.21.15.169:0/2195058044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 msgr2=0x7f79cc102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:34.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.973+0000 7f79d25da700 1 --2- 172.21.15.169:0/2195058044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 0x7f79cc102eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009910 tx=0x7f79bc009c20 comp rx=0 tx=0).stop 2024-04-25T10:46:34.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.974+0000 7f79d25da700 1 -- 172.21.15.169:0/2195058044 shutdown_connections 2024-04-25T10:46:34.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.974+0000 7f79d25da700 1 --2- 172.21.15.169:0/2195058044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 0x7f79cc102eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:34.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.974+0000 7f79d25da700 1 -- 172.21.15.169:0/2195058044 >> 172.21.15.169:0/2195058044 conn(0x7f79cc0fe0c0 msgr2=0x7f79cc100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.974+0000 7f79d25da700 1 -- 172.21.15.169:0/2195058044 shutdown_connections 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.974+0000 7f79d25da700 1 -- 172.21.15.169:0/2195058044 wait complete. 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.974+0000 7f79d25da700 1 Processor -- start 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.974+0000 7f79d25da700 1 -- start start 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79d25da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 0x7f79cc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79d25da700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc198c70 0x7f79cc19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79d25da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79cc19e140 0x7f79cc1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79d25da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc108a60 con 0x7f79cc102aa0 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79d25da700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1088e0 con 0x7f79cc19e140 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79d25da700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc108be0 con 0x7f79cc198c70 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc198c70 0x7f79cc19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc198c70 0x7f79cc19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56976/0 (socket says 172.21.15.169:56976) 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 -- 172.21.15.169:0/1293855620 learned_addr learned my addr 172.21.15.169:0/1293855620 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79d0b77700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79cc19e140 0x7f79cc1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cbfff700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 0x7f79cc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 -- 172.21.15.169:0/1293855620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79cc19e140 msgr2=0x7f79cc1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79cc19e140 0x7f79cc1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 -- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 msgr2=0x7f79cc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 0x7f79cc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.975+0000 7f79cb7fe700 1 -- 172.21.15.169:0/1293855620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc1a0ad0 con 0x7f79cc198c70 2024-04-25T10:46:34.981 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.976+0000 7f79cbfff700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 0x7f79cc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.976+0000 7f79cb7fe700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc198c70 0x7f79cc19dc00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b780 tx=0x7f79c000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.976+0000 7f79c97fa700 1 -- 172.21.15.169:0/1293855620 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0004230 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.976+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc1a0dc0 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.976+0000 7f79c97fa700 1 -- 172.21.15.169:0/1293855620 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c00043d0 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.976+0000 7f79c97fa700 1 -- 172.21.15.169:0/1293855620 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0011ad0 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.976+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc1a12f0 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.977+0000 7f79c97fa700 1 -- 172.21.15.169:0/1293855620 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f79c0011c90 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.977+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79cc1a1580 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.977+0000 7f79c97fa700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f79b403daa0 0x7f79b403ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.977+0000 7f79c97fa700 1 -- 172.21.15.169:0/1293855620 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f79c0051110 con 0x7f79cc198c70 2024-04-25T10:46:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.978+0000 7f79cbfff700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f79b403daa0 0x7f79b403ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:34.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.978+0000 7f79cbfff700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f79b403daa0 0x7f79b403ffb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79bc000c00 tx=0x7f79bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:34.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:34.983+0000 7f79c97fa700 1 -- 172.21.15.169:0/1293855620 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79c0020740 con 0x7f79cc198c70 2024-04-25T10:46:35.193 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.191+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi112=y;smithi169=z", "target": ["mon-mgr", ""]}) -- 0x7f79cc0fff00 con 0x7f79b403daa0 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: Reconfiguring mon.b (monmap changed)... 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:35.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:34 smithi169 ceph-mon[33049]: Reconfiguring daemon mon.b on smithi112 2024-04-25T10:46:35.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.199+0000 7f79c97fa700 1 -- 172.21.15.169:0/1293855620 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f79cc0fff00 con 0x7f79b403daa0 2024-04-25T10:46:35.200 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mgr update... 2024-04-25T10:46:35.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f79b403daa0 msgr2=0x7f79b403ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:35.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f79b403daa0 0x7f79b403ffb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79bc000c00 tx=0x7f79bc005fb0 comp rx=0 tx=0).stop 2024-04-25T10:46:35.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc198c70 msgr2=0x7f79cc19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:35.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc198c70 0x7f79cc19dc00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b780 tx=0x7f79c000ba90 comp rx=0 tx=0).stop 2024-04-25T10:46:35.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 shutdown_connections 2024-04-25T10:46:35.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f79b403daa0 0x7f79b403ffb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:35.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79cc19e140 0x7f79cc1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:35.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79cc102aa0 0x7f79cc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:35.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 --2- 172.21.15.169:0/1293855620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc198c70 0x7f79cc19dc00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:35.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 >> 172.21.15.169:0/1293855620 conn(0x7f79cc0fe0c0 msgr2=0x7f79cc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:35.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 shutdown_connections 2024-04-25T10:46:35.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:46:35.201+0000 7f79d25da700 1 -- 172.21.15.169:0/1293855620 wait complete. 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: Reconfiguring mon.b (monmap changed)... 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:34 smithi112 ceph-mon[32799]: Reconfiguring daemon mon.b on smithi112 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: Reconfiguring mon.b (monmap changed)... 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:35.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:34 smithi019 ceph-mon[29209]: Reconfiguring daemon mon.b on smithi112 2024-04-25T10:46:35.803 DEBUG:teuthology.orchestra.run.smithi112:mgr.y> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T10:46:35.805 DEBUG:teuthology.orchestra.run.smithi169:mgr.z> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T10:46:35.808 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T10:46:35.808 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:46:35.808 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:46:35.827 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:46:35.828 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-04-25T10:46:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:46:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:35.887 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T10:46:35.887 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:35.887 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:35.887 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 10:45:42.964265287 +0000 2024-04-25T10:46:35.887 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 10:40:39.689029102 +0000 2024-04-25T10:46:35.888 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 10:40:39.689029102 +0000 2024-04-25T10:46:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 10:40:39.689029102 +0000 2024-04-25T10:46:35.888 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:46:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:46:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:46:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000233133 s, 2.2 MB/s 2024-04-25T10:46:35.958 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:46:36.017 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-04-25T10:46:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:46:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:36.075 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T10:46:36.075 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:36.075 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:36.075 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 10:45:42.998264977 +0000 2024-04-25T10:46:36.075 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 10:40:40.052025892 +0000 2024-04-25T10:46:36.076 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 10:40:40.052025892 +0000 2024-04-25T10:46:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 10:40:40.052025892 +0000 2024-04-25T10:46:36.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:46:36.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:46:36.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:46:36.142 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000222755 s, 2.3 MB/s 2024-04-25T10:46:36.143 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:46:36.201 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 10:45:43.032264667 +0000 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 10:40:40.404022778 +0000 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 10:40:40.404022778 +0000 2024-04-25T10:46:36.258 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 10:40:40.404022778 +0000 2024-04-25T10:46:36.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:46:36.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi112=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: Saving service mgr spec with placement smithi019=x;smithi112=y;smithi169=z;count:3 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: Reconfiguring mon.c (monmap changed)... 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:36 smithi019 ceph-mon[29209]: Reconfiguring daemon mon.c on smithi169 2024-04-25T10:46:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:46:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:46:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000209828 s, 2.4 MB/s 2024-04-25T10:46:36.324 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:46:36.381 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi112=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: Saving service mgr spec with placement smithi019=x;smithi112=y;smithi169=z;count:3 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: Reconfiguring mon.c (monmap changed)... 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:36.417 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:36.418 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:36 smithi169 ceph-mon[33049]: Reconfiguring daemon mon.c on smithi169 2024-04-25T10:46:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:46:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T10:46:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 10:45:43.065264366 +0000 2024-04-25T10:46:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 10:40:40.768019559 +0000 2024-04-25T10:46:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 10:40:40.768019559 +0000 2024-04-25T10:46:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 10:40:40.768019559 +0000 2024-04-25T10:46:36.439 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:46:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:46:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:46:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000369463 s, 1.4 MB/s 2024-04-25T10:46:36.505 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:46:36.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi112=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:36.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: Saving service mgr spec with placement smithi019=x;smithi112=y;smithi169=z;count:3 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: Reconfiguring mon.c (monmap changed)... 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:36 smithi112 ceph-mon[32799]: Reconfiguring daemon mon.c on smithi169 2024-04-25T10:46:36.562 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:46:36.562 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:46:36.581 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:46:36.581 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-25T10:46:36.637 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:46:36.637 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:36.637 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-25T10:46:36.637 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:36.638 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:36.638 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 10:46:23.081701332 +0000 2024-04-25T10:46:36.638 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 10:41:09.354022845 +0000 2024-04-25T10:46:36.638 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 10:41:09.354022845 +0000 2024-04-25T10:46:36.638 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 10:41:09.354022845 +0000 2024-04-25T10:46:36.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:46:36.703 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T10:46:36.703 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T10:46:36.704 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00030762 s, 1.7 MB/s 2024-04-25T10:46:36.705 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:46:36.763 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-25T10:46:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:46:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:36.820 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T10:46:36.820 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:36.820 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:36.820 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 10:46:23.081701332 +0000 2024-04-25T10:46:36.820 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 10:41:09.693013397 +0000 2024-04-25T10:46:36.821 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 10:41:09.693013397 +0000 2024-04-25T10:46:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 10:41:09.693013397 +0000 2024-04-25T10:46:36.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:46:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T10:46:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T10:46:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000254806 s, 2.0 MB/s 2024-04-25T10:46:36.886 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:46:36.943 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-25T10:46:36.999 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:46:36.999 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:36.999 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T10:46:36.999 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:36.999 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:36.999 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 10:46:23.081701332 +0000 2024-04-25T10:46:37.000 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 10:41:10.037003809 +0000 2024-04-25T10:46:37.000 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 10:41:10.037003809 +0000 2024-04-25T10:46:37.000 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 10:41:10.037003809 +0000 2024-04-25T10:46:37.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:46:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T10:46:37.063 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T10:46:37.063 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00021105 s, 2.4 MB/s 2024-04-25T10:46:37.064 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:46:37.120 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-25T10:46:37.175 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:46:37.175 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:37.175 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T10:46:37.175 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:37.176 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:37.176 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 10:46:23.082701305 +0000 2024-04-25T10:46:37.176 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 10:41:10.373994417 +0000 2024-04-25T10:46:37.176 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 10:41:10.373994417 +0000 2024-04-25T10:46:37.176 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 10:41:10.373994417 +0000 2024-04-25T10:46:37.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:46:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T10:46:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T10:46:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000219695 s, 2.3 MB/s 2024-04-25T10:46:37.239 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:46:37.295 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:46:37.295 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:46:37.312 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:46:37.312 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-25T10:46:37.369 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:46:13.833689619 +0000 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:41:09.954303777 +0000 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:41:09.954303777 +0000 2024-04-25T10:46:37.370 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:41:09.954303777 +0000 2024-04-25T10:46:37.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:46:37.434 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:46:37.435 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:46:37.435 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000257649 s, 2.0 MB/s 2024-04-25T10:46:37.436 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:46:37.493 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:46:13.834689597 +0000 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:41:10.312295253 +0000 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:41:10.312295253 +0000 2024-04-25T10:46:37.549 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:41:10.312295253 +0000 2024-04-25T10:46:37.549 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:46:37.612 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:46:37.612 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:46:37.612 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00022493 s, 2.3 MB/s 2024-04-25T10:46:37.614 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:46:37.671 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-25T10:46:37.727 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:46:37.727 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:37.728 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T10:46:37.728 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:37.728 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:37.728 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:46:13.834689597 +0000 2024-04-25T10:46:37.728 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:41:10.650287206 +0000 2024-04-25T10:46:37.728 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:41:10.650287206 +0000 2024-04-25T10:46:37.728 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:41:10.650287206 +0000 2024-04-25T10:46:37.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:46:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:37 smithi169 ceph-mon[33049]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:37.775 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:46:37.775 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:46:37.775 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00021541 s, 2.4 MB/s 2024-04-25T10:46:37.776 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:46:37.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:37 smithi112 ceph-mon[32799]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:37.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:37.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:37.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:37.831 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-25T10:46:37.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:37 smithi019 ceph-mon[29209]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:46:13.835689574 +0000 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:41:11.020278397 +0000 2024-04-25T10:46:37.886 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:41:11.020278397 +0000 2024-04-25T10:46:37.887 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:41:11.020278397 +0000 2024-04-25T10:46:37.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:46:37.948 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:46:37.948 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:46:37.948 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000181225 s, 2.8 MB/s 2024-04-25T10:46:37.950 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:46:38.005 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-04-25T10:46:38.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T10:46:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:46:38.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:38 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:38.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:38 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:38.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:38 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:38.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:38 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:38.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:38 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:46:38.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:38 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:38.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:38 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:38.707 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:38 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:38.707 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:38 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:38.707 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:38 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:38.707 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:38 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:38.707 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:38 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:46:38.707 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:38 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:38.707 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:38 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:38 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:38 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:38.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:38 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:38.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:38 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:38.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:38 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:46:38.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:38 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:38.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:38 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:39.564 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:39 smithi169 ceph-mon[33049]: Deploying daemon mgr.z on smithi169 2024-04-25T10:46:39.564 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:39 smithi169 ceph-mon[33049]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:39.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:39 smithi112 ceph-mon[32799]: Deploying daemon mgr.z on smithi169 2024-04-25T10:46:39.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:39 smithi112 ceph-mon[32799]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:39 smithi019 ceph-mon[29209]: Deploying daemon mgr.z on smithi169 2024-04-25T10:46:39.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:39 smithi019 ceph-mon[29209]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:39.940 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:39 smithi169 systemd[1]: Starting Ceph mgr.z for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:46:39.950 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T10:46:39.950 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T10:46:39.950 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T10:46:39.950 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T10:46:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116307 s, 902 MB/s 2024-04-25T10:46:39.951 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 2024-04-25T10:46:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T10:46:40.210 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:39 smithi169 podman[34890]: 2024-04-25 10:46:39.841951499 +0000 UTC m=+0.017775078 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:40.210 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:39 smithi169 podman[34890]: 2024-04-25 10:46:39.944216239 +0000 UTC m=+0.120039846 container create 43f2139ac85b33cad3f7f381c8489809bba13ec3ae7c5a1f578ca9baed59d89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T10:46:40.210 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 podman[34890]: 2024-04-25 10:46:40.171876237 +0000 UTC m=+0.347699797 container init 43f2139ac85b33cad3f7f381c8489809bba13ec3ae7c5a1f578ca9baed59d89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T10:46:40.210 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 podman[34890]: 2024-04-25 10:46:40.174478291 +0000 UTC m=+0.350301851 container start 43f2139ac85b33cad3f7f381c8489809bba13ec3ae7c5a1f578ca9baed59d89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:46:40.210 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 ceph-mgr[34909]: -- 172.21.15.169:0/2374739129 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc2b52a5a0 con 0x55bc302b4000 2024-04-25T10:46:40.502 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 bash[34890]: 43f2139ac85b33cad3f7f381c8489809bba13ec3ae7c5a1f578ca9baed59d89a 2024-04-25T10:46:40.502 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 systemd[1]: Started Ceph mgr.z for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:46:40.502 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:40.340+0000 7fb65d78c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:46:40.565 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-04-25T10:46:40.811 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:46:40.940 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:40.527+0000 7fb65d78c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:46:40.940 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:40.629+0000 7fb65d78c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:46:40.940 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:40.938+0000 7fb65d78c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:46:41.252 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:41.023+0000 7fb65d78c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:41.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:41 smithi019 ceph-mon[29209]: Deploying daemon mgr.y on smithi112 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:41.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:41 smithi112 ceph-mon[32799]: Deploying daemon mgr.y on smithi112 2024-04-25T10:46:41.621 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:41.621 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.621 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.621 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:41.621 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:41.621 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:46:41.622 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:41.622 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:41.622 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:41 smithi169 ceph-mon[33049]: Deploying daemon mgr.y on smithi112 2024-04-25T10:46:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 -- 172.21.15.19:0/2636553238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103120 msgr2=0x7f58a4103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/2636553238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103120 0x7f58a4103530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58940098f0 tx=0x7f5894009c00 comp rx=0 tx=0).stop 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 -- 172.21.15.19:0/2636553238 shutdown_connections 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/2636553238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4104bf0 0x7f58a410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/2636553238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4103120 0x7f58a4103530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/2636553238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a4104250 0x7f58a4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 -- 172.21.15.19:0/2636553238 >> 172.21.15.19:0/2636553238 conn(0x7f58a40fe760 msgr2=0x7f58a4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 -- 172.21.15.19:0/2636553238 shutdown_connections 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 -- 172.21.15.19:0/2636553238 wait complete. 2024-04-25T10:46:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 Processor -- start 2024-04-25T10:46:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.244+0000 7f58aa0bd700 1 -- start start 2024-04-25T10:46:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58aa0bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a4103120 0x7f58a4076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58aa0bd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4104250 0x7f58a419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58aa0bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4104bf0 0x7f58a41a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58aa0bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410eb90 con 0x7f58a4104bf0 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58aa0bd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410ea10 con 0x7f58a4103120 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58aa0bd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410ed10 con 0x7f58a4104250 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4104bf0 0x7f58a41a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a4103120 0x7f58a4076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4104250 0x7f58a419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4104bf0 0x7f58a41a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40212/0 (socket says 172.21.15.19:40212) 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 -- 172.21.15.19:0/3515928094 learned_addr learned my addr 172.21.15.19:0/3515928094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 -- 172.21.15.19:0/3515928094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4104250 msgr2=0x7f58a419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4104250 0x7f58a419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:42.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 -- 172.21.15.19:0/3515928094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a4103120 msgr2=0x7f58a4076340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:46:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a4103120 0x7f58a4076340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:46:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 -- 172.21.15.19:0/3515928094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a41a23c0 con 0x7f58a4104bf0 2024-04-25T10:46:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.245+0000 7f58a3fff700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4104bf0 0x7f58a41a1b90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c110 tx=0x7f589c00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.246+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c0092e0 con 0x7f58a4104bf0 2024-04-25T10:46:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.246+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f589c004bb0 con 0x7f58a4104bf0 2024-04-25T10:46:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.246+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c007be0 con 0x7f58a4104bf0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.246+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a41a2650 con 0x7f58a4104bf0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.246+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a2b40 con 0x7f58a4104bf0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.247+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f589c01a020 con 0x7f58a4104bf0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.247+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a4051190 con 0x7f58a4104bf0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.247+0000 7f58a0ff9700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f588c03d850 0x7f588c03fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.247+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f589c0510d0 con 0x7f58a4104bf0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.247+0000 7f58a37fe700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f588c03d850 0x7f588c03fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.247+0000 7f58a37fe700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f588c03d850 0x7f588c03fd60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5894000c00 tx=0x7f5894019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:46:42.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.252+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f589c010040 con 0x7f58a4104bf0 2024-04-25T10:46:42.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:42.469+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f58a4063b90 con 0x7f588c03d850 2024-04-25T10:46:42.560 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:42 smithi112 podman[34165]: 2024-04-25 10:46:42.42610359 +0000 UTC m=+0.017756084 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:42.580 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:42.270+0000 7fb65d78c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:46:42.580 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:42.447+0000 7fb65d78c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:46:42.863 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:42 smithi112 podman[34165]: 2024-04-25 10:46:42.696027139 +0000 UTC m=+0.287679613 container init 9d341b766b7d6a3fcc4f075ae315d73cea4e9dd22d58cfafebb482b680abf10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:46:42.863 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:42 smithi112 podman[34165]: 2024-04-25 10:46:42.698560554 +0000 UTC m=+0.290213024 container start 9d341b766b7d6a3fcc4f075ae315d73cea4e9dd22d58cfafebb482b680abf10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T10:46:42.864 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:42 smithi112 ceph-mgr[34184]: -- 172.21.15.112:0/3428735657 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fc68be5a0 con 0x561fcb648000 2024-04-25T10:46:42.864 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:42 smithi112 bash[34165]: 9d341b766b7d6a3fcc4f075ae315d73cea4e9dd22d58cfafebb482b680abf10a 2024-04-25T10:46:42.864 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:42 smithi112 systemd[1]: Started Ceph mgr.y for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:46:42.864 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:42.861+0000 7fafb4528200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:46:42.918 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:42.624+0000 7fb65d78c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:46:42.919 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:42.917+0000 7fb65d78c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:46:43.160 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:43.047+0000 7fafb4528200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:46:43.161 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:43.159+0000 7fafb4528200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:46:43.226 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:43.011+0000 7fb65d78c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:46:43.227 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:43.225+0000 7fb65d78c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.494 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.495 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.495 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:43.428+0000 7fb65d78c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:43.809 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:43.492+0000 7fafb4528200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:46:43.810 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:43.598+0000 7fafb4528200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:46:44.252 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:44.075+0000 7fb65d78c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:46:44.819 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:44.517+0000 7fb65d78c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:46:44.819 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:44.616+0000 7fb65d78c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:46:44.819 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:44.816+0000 7fb65d78c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:46:45.181 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:44 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:44.843+0000 7fafb4528200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:46:45.181 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:45.011+0000 7fafb4528200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:46:45.181 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:45.179+0000 7fafb4528200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:46:45.467 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:45.467 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.467 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.467 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.467 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.467 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2554078969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1a36b37-3389-4f73-873b-7dd279f1701a"}]: dispatch 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2554078969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1a36b37-3389-4f73-873b-7dd279f1701a"}]': finished 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T10:46:45.468 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:45.348+0000 7fb65d78c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:45.442+0000 7fb65d78c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2554078969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1a36b37-3389-4f73-873b-7dd279f1701a"}]: dispatch 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2554078969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1a36b37-3389-4f73-873b-7dd279f1701a"}]': finished 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T10:46:45.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:46:45.758 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:45.465+0000 7fafb4528200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:46:45.758 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:45.556+0000 7fafb4528200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:46:45.758 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:45.756+0000 7fafb4528200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:45.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2554078969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1a36b37-3389-4f73-873b-7dd279f1701a"}]: dispatch 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2554078969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1a36b37-3389-4f73-873b-7dd279f1701a"}]': finished 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T10:46:45.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:46:46.002 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:45.687+0000 7fb65d78c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:46:46.002 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:45.822+0000 7fb65d78c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:46:46.002 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:45.917+0000 7fb65d78c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:46:46.059 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:45.947+0000 7fafb4528200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:46:46.340 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:46.032+0000 7fb65d78c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:46:46.340 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:46.338+0000 7fb65d78c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:46:46.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:46 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1433127798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:46:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:46 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1433127798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:46:46.810 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:46.550+0000 7fafb4528200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:46:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:46 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1433127798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:46:47.252 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:46.940+0000 7fb65d78c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:46:47.252 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:46:47 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:46:47.155+0000 7fb65d78c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:46:47.266 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:46.970+0000 7fafb4528200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:46:47.266 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:47.064+0000 7fafb4528200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:46:47.266 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:47.265+0000 7fafb4528200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:46:47.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:47.383+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) ==== 98518+0+0 (secure 0 0 0) 0x7f589c012070 con 0x7f58a4104bf0 2024-04-25T10:46:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-mon[32799]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:47.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-mon[32799]: Standby manager daemon z started 2024-04-25T10:46:47.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:46:47.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:46:47.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:46:47.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:46:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:47 smithi169 ceph-mon[33049]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:47 smithi169 ceph-mon[33049]: Standby manager daemon z started 2024-04-25T10:46:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:47 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:46:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:47 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:46:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:47 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:46:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:47 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:46:47.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:47 smithi019 ceph-mon[29209]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:47 smithi019 ceph-mon[29209]: Standby manager daemon z started 2024-04-25T10:46:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:47 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:46:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:47 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:46:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:47 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:46:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:47 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2450588837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:46:48.059 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:47.785+0000 7fafb4528200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:46:48.060 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:47.876+0000 7fafb4528200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:46:48.386 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:48.112+0000 7fafb4528200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:46:48.386 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:48.242+0000 7fafb4528200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:46:48.386 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:48.336+0000 7fafb4528200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:46:48.750 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:48.452+0000 7fafb4528200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:46:48.750 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:48 smithi112 ceph-mon[32799]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T10:46:48.750 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:48 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:46:48.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:48 smithi169 ceph-mon[33049]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T10:46:48.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:48 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:46:48.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:48 smithi019 ceph-mon[29209]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T10:46:48.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:48 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:46:49.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:49 smithi019 ceph-mon[29209]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:49.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:49 smithi169 ceph-mon[33049]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:49.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:49 smithi112 ceph-mon[32799]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:49.810 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:48.749+0000 7fafb4528200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:46:49.810 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:49.336+0000 7fafb4528200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:46:49.810 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:46:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:46:49.549+0000 7fafb4528200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:46:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:46:50.399+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f589c01a310 con 0x7f58a4104bf0 2024-04-25T10:46:50.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:50 smithi169 ceph-mon[33049]: Standby manager daemon y started 2024-04-25T10:46:50.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:50 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:46:50.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:50 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:46:50.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:50 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:46:50.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:50 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:46:50.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:50 smithi019 ceph-mon[29209]: Standby manager daemon y started 2024-04-25T10:46:50.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:50 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:46:50.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:50 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:46:50.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:50 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:46:50.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:50 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:46:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:50 smithi112 ceph-mon[32799]: Standby manager daemon y started 2024-04-25T10:46:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:50 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:46:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:50 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:46:50.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:50 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:46:50.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:50 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3354944915' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:51.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:51.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:46:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:53.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:53 smithi019 ceph-mon[29209]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:53 smithi169 ceph-mon[33049]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:53.786 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:53 smithi112 ceph-mon[32799]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:54.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T10:46:54.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:54.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:54 smithi019 ceph-mon[29209]: Deploying daemon osd.0 on smithi019 2024-04-25T10:46:54.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T10:46:54.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:54.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:54 smithi169 ceph-mon[33049]: Deploying daemon osd.0 on smithi019 2024-04-25T10:46:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T10:46:54.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:46:54.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:54 smithi112 ceph-mon[32799]: Deploying daemon osd.0 on smithi019 2024-04-25T10:46:55.748 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:55 smithi019 ceph-mon[29209]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:55.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:55 smithi169 ceph-mon[33049]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:55 smithi112 ceph-mon[32799]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:57 smithi019 ceph-mon[29209]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:57 smithi169 ceph-mon[33049]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:57.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:57 smithi112 ceph-mon[32799]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:59.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:59 smithi169 ceph-mon[33049]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:59.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:59 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:59.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:59 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:59.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:46:59 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:59.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:59 smithi019 ceph-mon[29209]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:59.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:59.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:59.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:46:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:46:59.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:59 smithi112 ceph-mon[32799]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:46:59.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:59 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:59.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:59 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:46:59.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:46:59 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:01.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:01 smithi019 ceph-mon[29209]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:01.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:01 smithi169 ceph-mon[33049]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:01.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:01 smithi112 ceph-mon[32799]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:02.581 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-04-25T10:47:02.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.580+0000 7f58a0ff9700 1 -- 172.21.15.19:0/3515928094 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f58a4063b90 con 0x7f588c03d850 2024-04-25T10:47:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.582+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f588c03d850 msgr2=0x7f588c03fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.582+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f588c03d850 0x7f588c03fd60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5894000c00 tx=0x7f5894019040 comp rx=0 tx=0).stop 2024-04-25T10:47:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.582+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4104bf0 msgr2=0x7f58a41a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.582+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4104bf0 0x7f58a41a1b90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c110 tx=0x7f589c00c420 comp rx=0 tx=0).stop 2024-04-25T10:47:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 shutdown_connections 2024-04-25T10:47:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f588c03d850 0x7f588c03fd60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a4104bf0 0x7f58a41a1b90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a4103120 0x7f58a4076340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 --2- 172.21.15.19:0/3515928094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4104250 0x7f58a419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 >> 172.21.15.19:0/3515928094 conn(0x7f58a40fe760 msgr2=0x7f58a4107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:47:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 shutdown_connections 2024-04-25T10:47:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:02.583+0000 7f58aa0bd700 1 -- 172.21.15.19:0/3515928094 wait complete. 2024-04-25T10:47:03.279 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T10:47:03.282 INFO:tasks.cephadm:Deploying osd.1 on smithi112 with /dev/vg_nvme/lv_4... 2024-04-25T10:47:03.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T10:47:03.526 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:47:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:03 smithi112 ceph-mon[32799]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:03 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:03 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:03 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:03 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:03 smithi112 ceph-mon[32799]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T10:47:03.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:03 smithi019 ceph-mon[29209]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:03.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:03 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:03 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:03 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:03 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:03.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:03 smithi019 ceph-mon[29209]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T10:47:04.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:03 smithi169 ceph-mon[33049]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:04.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:03 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:04.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:03 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:04.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:03 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:04.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:03 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:04.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:03 smithi169 ceph-mon[33049]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T10:47:04.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:04 smithi019 ceph-mon[29209]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T10:47:04.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:04 smithi019 ceph-mon[29209]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T10:47:04.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:04 smithi019 ceph-mon[29209]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:47:04.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:04 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:04.776 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:04 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0[43995]: 2024-04-25T10:47:04.675+0000 7fcaf5098700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:47:04.777 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:04 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0[43995]: 2024-04-25T10:47:04.683+0000 7fcaf0e4d700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:47:04.986 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:04 smithi112 ceph-mon[32799]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T10:47:04.987 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:04 smithi112 ceph-mon[32799]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T10:47:04.987 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:04 smithi112 ceph-mon[32799]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:47:04.987 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:04 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:05.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:04 smithi169 ceph-mon[33049]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T10:47:05.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:04 smithi169 ceph-mon[33049]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T10:47:05.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:04 smithi169 ceph-mon[33049]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:47:05.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:04 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T10:47:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T10:47:05.523 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T10:47:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T10:47:05.523 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113889 s, 921 MB/s 2024-04-25T10:47:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T10:47:05.913 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:05 smithi112 ceph-mon[32799]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:05.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:05 smithi112 ceph-mon[32799]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:47:05.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:05 smithi112 ceph-mon[32799]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T10:47:05.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:05.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:05.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:05 smithi169 ceph-mon[33049]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:06.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:05 smithi169 ceph-mon[33049]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:47:06.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:05 smithi169 ceph-mon[33049]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T10:47:06.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:05 smithi019 ceph-mon[29209]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:06.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:05 smithi019 ceph-mon[29209]: from='osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:47:06.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:05 smithi019 ceph-mon[29209]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T10:47:06.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-04-25T10:47:06.425 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:47:06.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] boot 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:06.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] boot 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:07.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:07.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:07.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:47:07.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: osd.0 [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] boot 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:07.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 -- 172.21.15.112:0/543084830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105340 msgr2=0x7ff19c105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 --2- 172.21.15.112:0/543084830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105340 0x7ff19c105750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff184009990 tx=0x7ff184009ca0 comp rx=0 tx=0).stop 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 -- 172.21.15.112:0/543084830 shutdown_connections 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 --2- 172.21.15.112:0/543084830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff19c105ce0 0x7ff19c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 --2- 172.21.15.112:0/543084830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c104210 0x7ff19c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 --2- 172.21.15.112:0/543084830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105340 0x7ff19c105750 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 -- 172.21.15.112:0/543084830 >> 172.21.15.112:0/543084830 conn(0x7ff19c0ff870 msgr2=0x7ff19c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 -- 172.21.15.112:0/543084830 shutdown_connections 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.895+0000 7ff1a1340700 1 -- 172.21.15.112:0/543084830 wait complete. 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 Processor -- start 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 -- start start 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff19c104210 0x7ff19c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c105340 0x7ff19c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105ce0 0x7ff19c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c10fcf0 con 0x7ff19c104210 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c10fb70 con 0x7ff19c105ce0 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1a1340700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c10fe70 con 0x7ff19c105340 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff19b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105ce0 0x7ff19c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff19b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105ce0 0x7ff19c1a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41260/0 (socket says 172.21.15.112:41260) 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff19b7fe700 1 -- 172.21.15.112:0/372283983 learned_addr learned my addr 172.21.15.112:0/372283983 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff1927fc700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c105340 0x7ff19c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff19affd700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff19c104210 0x7ff19c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff19b7fe700 1 -- 172.21.15.112:0/372283983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c105340 msgr2=0x7ff19c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.896+0000 7ff19b7fe700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c105340 0x7ff19c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff19b7fe700 1 -- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff19c104210 msgr2=0x7ff19c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff19b7fe700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff19c104210 0x7ff19c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff19b7fe700 1 -- 172.21.15.112:0/372283983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c1a2340 con 0x7ff19c105ce0 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff19b7fe700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105ce0 0x7ff19c1a1cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff194007ee0 tx=0x7ff19400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff198ff9700 1 -- 172.21.15.112:0/372283983 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194004bb0 con 0x7ff19c105ce0 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff198ff9700 1 -- 172.21.15.112:0/372283983 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff194004d50 con 0x7ff19c105ce0 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c1a2630 con 0x7ff19c105ce0 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff198ff9700 1 -- 172.21.15.112:0/372283983 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194005920 con 0x7ff19c105ce0 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.897+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1a2bf0 con 0x7ff19c105ce0 2024-04-25T10:47:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.898+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19c051190 con 0x7ff19c105ce0 2024-04-25T10:47:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.899+0000 7ff198ff9700 1 -- 172.21.15.112:0/372283983 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7ff19401a020 con 0x7ff19c105ce0 2024-04-25T10:47:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.900+0000 7ff198ff9700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff1880afb70 0x7ff1880b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.900+0000 7ff198ff9700 1 -- 172.21.15.112:0/372283983 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff1940dcec0 con 0x7ff19c105ce0 2024-04-25T10:47:07.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.900+0000 7ff19affd700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff1880afb70 0x7ff1880b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:07.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.901+0000 7ff19affd700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff1880afb70 0x7ff1880b2080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff19c105240 tx=0x7ff18c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:47:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:07.904+0000 7ff198ff9700 1 -- 172.21.15.112:0/372283983 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff194010070 con 0x7ff19c105ce0 2024-04-25T10:47:08.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:07 smithi169 ceph-mon[33049]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:08.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:07 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi019 2024-04-25T10:47:08.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:07 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-25T10:47:08.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:07 smithi169 ceph-mon[33049]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T10:47:08.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:08.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:08.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:08.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:07 smithi112 ceph-mon[32799]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:07 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi019 2024-04-25T10:47:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:07 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-25T10:47:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:07 smithi112 ceph-mon[32799]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T10:47:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:08.109+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff19c063ad0 con 0x7ff1880afb70 2024-04-25T10:47:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:07 smithi019 ceph-mon[29209]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:47:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:07 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi019 2024-04-25T10:47:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:07 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-25T10:47:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:07 smithi019 ceph-mon[29209]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T10:47:08.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:08.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:08.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:08.986 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:08.986 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:08.986 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:09.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:09.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:09.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:09.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:09.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:09.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:10.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:09 smithi169 ceph-mon[33049]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:10.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:09 smithi169 ceph-mon[33049]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:09 smithi112 ceph-mon[32799]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:09 smithi112 ceph-mon[32799]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:10.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:09 smithi019 ceph-mon[29209]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:10.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:09 smithi019 ceph-mon[29209]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:12.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:11 smithi169 ceph-mon[33049]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:12.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:11 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/430071011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]: dispatch 2024-04-25T10:47:12.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:11 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]: dispatch 2024-04-25T10:47:12.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:11 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]': finished 2024-04-25T10:47:12.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:11 smithi169 ceph-mon[33049]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T10:47:12.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:12.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:11 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/3795644394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:47:12.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:11 smithi112 ceph-mon[32799]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:12.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:11 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/430071011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]: dispatch 2024-04-25T10:47:12.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:11 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]: dispatch 2024-04-25T10:47:12.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:11 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]': finished 2024-04-25T10:47:12.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:11 smithi112 ceph-mon[32799]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T10:47:12.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:12.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:11 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/3795644394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:47:12.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:11 smithi019 ceph-mon[29209]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:12.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:11 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/430071011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]: dispatch 2024-04-25T10:47:12.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:11 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]: dispatch 2024-04-25T10:47:12.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:11 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d41a19a6-e1ab-4d69-87da-3ce650ec50c0"}]': finished 2024-04-25T10:47:12.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:11 smithi019 ceph-mon[29209]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T10:47:12.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:12.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:11 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/3795644394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:47:13.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:12 smithi169 ceph-mon[33049]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:13.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:12 smithi112 ceph-mon[32799]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:13.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:12 smithi019 ceph-mon[29209]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:15.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:14 smithi169 ceph-mon[33049]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:15.272 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:14 smithi112 ceph-mon[32799]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:15.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:14 smithi019 ceph-mon[29209]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:17.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:16 smithi019 ceph-mon[29209]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:17.187 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:16 smithi112 ceph-mon[32799]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:16 smithi169 ceph-mon[33049]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:19.128 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:18 smithi112 ceph-mon[32799]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:19.128 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:18 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T10:47:19.128 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:18 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:19.128 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:18 smithi112 ceph-mon[32799]: Deploying daemon osd.1 on smithi112 2024-04-25T10:47:19.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:18 smithi169 ceph-mon[33049]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:19.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:18 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T10:47:19.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:18 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:19.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:18 smithi169 ceph-mon[33049]: Deploying daemon osd.1 on smithi112 2024-04-25T10:47:19.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:18 smithi019 ceph-mon[29209]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:19.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:18 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T10:47:19.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:18 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:19.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:18 smithi019 ceph-mon[29209]: Deploying daemon osd.1 on smithi112 2024-04-25T10:47:21.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:20 smithi169 ceph-mon[33049]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:21.264 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:20 smithi112 ceph-mon[32799]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:20 smithi019 ceph-mon[29209]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:23.120 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:22 smithi112 ceph-mon[32799]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:23.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:22 smithi169 ceph-mon[33049]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:23.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:22 smithi019 ceph-mon[29209]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:25.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:25.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:25.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:25.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:25.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:25.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:25.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:25.797 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:25 smithi112 ceph-mon[32799]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:26.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:25 smithi169 ceph-mon[33049]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:26.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:25 smithi019 ceph-mon[29209]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:27.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:27 smithi112 ceph-mon[32799]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:27.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:27.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:27.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.533 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 1 on host 'smithi112' 2024-04-25T10:47:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.530+0000 7ff198ff9700 1 -- 172.21.15.112:0/372283983 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff19c063ad0 con 0x7ff1880afb70 2024-04-25T10:47:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff1880afb70 msgr2=0x7ff1880b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff1880afb70 0x7ff1880b2080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff19c105240 tx=0x7ff18c009190 comp rx=0 tx=0).stop 2024-04-25T10:47:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105ce0 msgr2=0x7ff19c1a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105ce0 0x7ff19c1a1cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff194007ee0 tx=0x7ff19400d3b0 comp rx=0 tx=0).stop 2024-04-25T10:47:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 shutdown_connections 2024-04-25T10:47:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff1880afb70 0x7ff1880b2080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c105ce0 0x7ff19c1a1cc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff19c104210 0x7ff19c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.533+0000 7ff1a1340700 1 --2- 172.21.15.112:0/372283983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c105340 0x7ff19c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.534+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 >> 172.21.15.112:0/372283983 conn(0x7ff19c0ff870 msgr2=0x7ff19c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:47:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.534+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 shutdown_connections 2024-04-25T10:47:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:47:27.534+0000 7ff1a1340700 1 -- 172.21.15.112:0/372283983 wait complete. 2024-04-25T10:47:27.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:27 smithi019 ceph-mon[29209]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:27.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:27.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:27.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:27 smithi169 ceph-mon[33049]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:27.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:27.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:27.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:27.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:28.187 DEBUG:teuthology.orchestra.run.smithi112:osd.1> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T10:47:28.189 INFO:tasks.cephadm:Deploying osd.2 on smithi169 with /dev/vg_nvme/lv_4... 2024-04-25T10:47:28.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T10:47:28.433 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:47:28.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:28.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:28.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:28.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:28 smithi169 ceph-mon[33049]: from='osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:47:28.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:28 smithi169 ceph-mon[33049]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:47:28.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:28 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:28.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:28 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:28.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:28 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:28.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:28 smithi112 ceph-mon[32799]: from='osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:47:28.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:28 smithi112 ceph-mon[32799]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:47:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:28 smithi019 ceph-mon[29209]: from='osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:47:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:28 smithi019 ceph-mon[29209]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:47:29.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:29 smithi169 ceph-mon[33049]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:29.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:29 smithi169 ceph-mon[33049]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T10:47:29.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:29 smithi169 ceph-mon[33049]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T10:47:29.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:29.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:29 smithi169 ceph-mon[33049]: from='osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:47:29.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:29 smithi169 ceph-mon[33049]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:47:29.775 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-mon[32799]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:29.775 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-mon[32799]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T10:47:29.775 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-mon[32799]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T10:47:29.775 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:29.775 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-mon[32799]: from='osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:47:29.775 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-mon[32799]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:47:29.776 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1[37412]: 2024-04-25T10:47:29.507+0000 7f2c78ab0700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:47:29.776 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1[37412]: 2024-04-25T10:47:29.511+0000 7f2c74865700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:47:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:29 smithi019 ceph-mon[29209]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:29 smithi019 ceph-mon[29209]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T10:47:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:29 smithi019 ceph-mon[29209]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T10:47:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:29 smithi019 ceph-mon[29209]: from='osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:47:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:29 smithi019 ceph-mon[29209]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:47:30.245 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T10:47:30.245 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T10:47:30.246 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T10:47:30.246 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T10:47:30.246 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011487 s, 913 MB/s 2024-04-25T10:47:30.246 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T10:47:30.592 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:30 smithi169 ceph-mon[33049]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:47:30.592 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:30 smithi169 ceph-mon[33049]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T10:47:30.593 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:30.593 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:30.694 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:30 smithi112 ceph-mon[32799]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:47:30.694 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:30 smithi112 ceph-mon[32799]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T10:47:30.695 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:30.695 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:30 smithi019 ceph-mon[29209]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:47:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:30 smithi019 ceph-mon[29209]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T10:47:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:30.886 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-04-25T10:47:31.142 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] boot 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:31.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:31.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:31.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] boot 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:31.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:47:31.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: osd.1 [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] boot 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:31.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:32.618 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 -- 172.21.15.169:0/2632444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128103120 msgr2=0x7f0128103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:32.618 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 --2- 172.21.15.169:0/2632444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128103120 0x7f0128103530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0124009990 tx=0x7f0124009ca0 comp rx=0 tx=0).stop 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 -- 172.21.15.169:0/2632444419 shutdown_connections 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 --2- 172.21.15.169:0/2632444419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0128104bf0 0x7f012810b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 --2- 172.21.15.169:0/2632444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128103120 0x7f0128103530 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 --2- 172.21.15.169:0/2632444419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128104250 0x7f0128104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 -- 172.21.15.169:0/2632444419 >> 172.21.15.169:0/2632444419 conn(0x7f01280fe760 msgr2=0x7f0128100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 -- 172.21.15.169:0/2632444419 shutdown_connections 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.614+0000 7f012f283700 1 -- 172.21.15.169:0/2632444419 wait complete. 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 Processor -- start 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 -- start start 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128103120 0x7f0128106ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0128104250 0x7f0128105590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128104bf0 0x7f0128105ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f012810eb60 con 0x7f0128103120 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f012810e9e0 con 0x7f0128104250 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012f283700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f012810ece0 con 0x7f0128104bf0 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012d820700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128104bf0 0x7f0128105ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012d820700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128104bf0 0x7f0128105ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54062/0 (socket says 172.21.15.169:54062) 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012d820700 1 -- 172.21.15.169:0/621324786 learned_addr learned my addr 172.21.15.169:0/621324786 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012d01f700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128103120 0x7f0128106ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.615+0000 7f012c81e700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0128104250 0x7f0128105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012d820700 1 -- 172.21.15.169:0/621324786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0128104250 msgr2=0x7f0128105590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012d820700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0128104250 0x7f0128105590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012d820700 1 -- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128103120 msgr2=0x7f0128106ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012d820700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128103120 0x7f0128106ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012d820700 1 -- 172.21.15.169:0/621324786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0128106230 con 0x7f0128104bf0 2024-04-25T10:47:32.620 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012d820700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128104bf0 0x7f0128105ad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f012000ada0 tx=0x7f012000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:47:32.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f011e7fc700 1 -- 172.21.15.169:0/621324786 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120017020 con 0x7f0128104bf0 2024-04-25T10:47:32.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01281a4910 con 0x7f0128104bf0 2024-04-25T10:47:32.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f011e7fc700 1 -- 172.21.15.169:0/621324786 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012000cd80 con 0x7f0128104bf0 2024-04-25T10:47:32.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f011e7fc700 1 -- 172.21.15.169:0/621324786 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120005260 con 0x7f0128104bf0 2024-04-25T10:47:32.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.616+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01281a4da0 con 0x7f0128104bf0 2024-04-25T10:47:32.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.617+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0128051190 con 0x7f0128104bf0 2024-04-25T10:47:32.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.618+0000 7f011e7fc700 1 -- 172.21.15.169:0/621324786 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f012000a090 con 0x7f0128104bf0 2024-04-25T10:47:32.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.619+0000 7f011e7fc700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01140afaf0 0x7f01140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:32.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.619+0000 7f011e7fc700 1 -- 172.21.15.169:0/621324786 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f01200dd090 con 0x7f0128104bf0 2024-04-25T10:47:32.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.619+0000 7f012d01f700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01140afaf0 0x7f01140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:32.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.620+0000 7f012d01f700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01140afaf0 0x7f01140b2000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01240038a0 tx=0x7f012401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:47:32.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.623+0000 7f011e7fc700 1 -- 172.21.15.169:0/621324786 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0120021020 con 0x7f0128104bf0 2024-04-25T10:47:32.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:32 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi112 2024-04-25T10:47:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:32 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T10:47:32.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:32 smithi169 ceph-mon[33049]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T10:47:32.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:32 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi112 2024-04-25T10:47:32.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:32 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T10:47:32.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:32 smithi112 ceph-mon[32799]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T10:47:32.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:32.832+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f01281007d0 con 0x7f01140afaf0 2024-04-25T10:47:32.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:32 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi112 2024-04-25T10:47:32.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:32 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T10:47:32.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:32 smithi019 ceph-mon[29209]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T10:47:33.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:33 smithi169 ceph-mon[33049]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:33.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:33.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:33.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:33 smithi112 ceph-mon[32799]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:33.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:33.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:33 smithi019 ceph-mon[29209]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:33.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:33.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:34.674 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:34 smithi169 ceph-mon[33049]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:34.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:34 smithi112 ceph-mon[32799]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:34.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:34 smithi019 ceph-mon[29209]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:35.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:35 smithi112 ceph-mon[32799]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:35 smithi019 ceph-mon[29209]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:35.885 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:35 smithi169 ceph-mon[33049]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:36.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:36 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/3368040318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]: dispatch 2024-04-25T10:47:36.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:36 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]: dispatch 2024-04-25T10:47:36.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:36 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]': finished 2024-04-25T10:47:36.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:36 smithi112 ceph-mon[32799]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T10:47:36.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:36.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:36 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/1766873835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:47:36.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:36 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/3368040318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]: dispatch 2024-04-25T10:47:36.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:36 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]: dispatch 2024-04-25T10:47:36.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:36 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]': finished 2024-04-25T10:47:36.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:36 smithi019 ceph-mon[29209]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T10:47:36.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:36.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:36 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/1766873835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:47:37.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:36 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/3368040318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]: dispatch 2024-04-25T10:47:37.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:36 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]: dispatch 2024-04-25T10:47:37.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:36 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d"}]': finished 2024-04-25T10:47:37.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:36 smithi169 ceph-mon[33049]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T10:47:37.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:37.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:36 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/1766873835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:47:37.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:37 smithi019 ceph-mon[29209]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:38.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:37 smithi169 ceph-mon[33049]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:38.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:37 smithi112 ceph-mon[32799]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:39.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:39 smithi169 ceph-mon[33049]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:39 smithi019 ceph-mon[29209]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:40.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:39 smithi112 ceph-mon[32799]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:41 smithi019 ceph-mon[29209]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:41.862 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:41 smithi112 ceph-mon[32799]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:41.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:41 smithi169 ceph-mon[33049]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:43.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:43 smithi019 ceph-mon[29209]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:43.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T10:47:43.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:43.880 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:43 smithi169 ceph-mon[33049]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:43.881 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T10:47:43.881 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:44.055 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:43 smithi112 ceph-mon[32799]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:44.055 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T10:47:44.055 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:44.759 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:44 smithi169 ceph-mon[33049]: Deploying daemon osd.2 on smithi169 2024-04-25T10:47:45.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:44 smithi112 ceph-mon[32799]: Deploying daemon osd.2 on smithi169 2024-04-25T10:47:45.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:44 smithi019 ceph-mon[29209]: Deploying daemon osd.2 on smithi169 2024-04-25T10:47:45.782 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:45 smithi169 ceph-mon[33049]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:46.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:45 smithi112 ceph-mon[32799]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:46.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:45 smithi019 ceph-mon[29209]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:47.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:47 smithi169 ceph-mon[33049]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:48.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:47 smithi112 ceph-mon[32799]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:48.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:47 smithi019 ceph-mon[29209]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:49.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:49 smithi169 ceph-mon[33049]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:49.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:49 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:49.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:49 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:49.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:49 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:49.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:49 smithi112 ceph-mon[32799]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:49.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:49 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:49.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:49 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:49.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:49 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:49.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:49 smithi019 ceph-mon[29209]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:49.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:49 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:49.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:49 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:49.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:49 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:51.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:51 smithi169 ceph-mon[33049]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:51.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:51.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:51.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:51 smithi112 ceph-mon[32799]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:51.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:51 smithi019 ceph-mon[29209]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:51.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:51.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:51.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:51.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:52.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.355+0000 7f011e7fc700 1 -- 172.21.15.169:0/621324786 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f01281007d0 con 0x7f01140afaf0 2024-04-25T10:47:52.357 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 2 on host 'smithi169' 2024-04-25T10:47:52.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01140afaf0 msgr2=0x7f01140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:52.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01140afaf0 0x7f01140b2000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01240038a0 tx=0x7f012401a040 comp rx=0 tx=0).stop 2024-04-25T10:47:52.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128104bf0 msgr2=0x7f0128105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128104bf0 0x7f0128105ad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f012000ada0 tx=0x7f012000c330 comp rx=0 tx=0).stop 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 shutdown_connections 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01140afaf0 0x7f01140b2000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0128104bf0 0x7f0128105ad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128103120 0x7f0128106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 --2- 172.21.15.169:0/621324786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0128104250 0x7f0128105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 >> 172.21.15.169:0/621324786 conn(0x7f01280fe760 msgr2=0x7f01281000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 shutdown_connections 2024-04-25T10:47:52.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:47:52.358+0000 7f012f283700 1 -- 172.21.15.169:0/621324786 wait complete. 2024-04-25T10:47:52.974 DEBUG:teuthology.orchestra.run.smithi169:osd.2> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T10:47:52.977 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-04-25T10:47:52.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T10:47:53.231 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:47:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:53 smithi169 ceph-mon[33049]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:53.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:53.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:53.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:53 smithi169 ceph-mon[33049]: from='osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:47:53.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:53 smithi169 ceph-mon[33049]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:47:53.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:53 smithi019 ceph-mon[29209]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:53.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:53.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:53.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:53.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:53 smithi019 ceph-mon[29209]: from='osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:47:53.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:53 smithi019 ceph-mon[29209]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:47:53.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:53 smithi112 ceph-mon[32799]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:53.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:53.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:53.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:47:53.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:53 smithi112 ceph-mon[32799]: from='osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:47:53.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:53 smithi112 ceph-mon[32799]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:47:54.480 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:54 smithi169 ceph-mon[33049]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T10:47:54.480 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:54 smithi169 ceph-mon[33049]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T10:47:54.480 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:54.480 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:54 smithi169 ceph-mon[33049]: from='osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:47:54.480 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:54 smithi169 ceph-mon[33049]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:47:54.480 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2[38210]: 2024-04-25T10:47:54.347+0000 7f401d4d9700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:47:54.481 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2[38210]: 2024-04-25T10:47:54.351+0000 7f401928e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:47:54.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:54 smithi112 ceph-mon[32799]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T10:47:54.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:54 smithi112 ceph-mon[32799]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T10:47:54.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:54.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:54 smithi112 ceph-mon[32799]: from='osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:47:54.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:54 smithi112 ceph-mon[32799]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:47:54.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:54 smithi019 ceph-mon[29209]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T10:47:54.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:54 smithi019 ceph-mon[29209]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T10:47:54.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:54.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:54 smithi019 ceph-mon[29209]: from='osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:47:54.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:54 smithi019 ceph-mon[29209]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:47:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T10:47:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T10:47:55.204 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T10:47:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T10:47:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113937 s, 920 MB/s 2024-04-25T10:47:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T10:47:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:55 smithi019 ceph-mon[29209]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:55 smithi019 ceph-mon[29209]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:47:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:55 smithi019 ceph-mon[29209]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T10:47:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:55.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:55.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:55 smithi169 ceph-mon[33049]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:55.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:55 smithi169 ceph-mon[33049]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:47:55.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:55 smithi169 ceph-mon[33049]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T10:47:55.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:55.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:55 smithi112 ceph-mon[32799]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:55 smithi112 ceph-mon[32799]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:47:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:55 smithi112 ceph-mon[32799]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T10:47:55.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:55.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:55.991 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-04-25T10:47:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:47:56.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] boot 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:56.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] boot 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:56.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:47:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:47:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: osd.2 [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] boot 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:56.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:47:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:57 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi169 2024-04-25T10:47:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:57 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T10:47:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:57 smithi019 ceph-mon[29209]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T10:47:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:57 smithi019 ceph-mon[29209]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T10:47:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:47:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:57 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi169 2024-04-25T10:47:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:57 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T10:47:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:57 smithi169 ceph-mon[33049]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T10:47:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:57 smithi169 ceph-mon[33049]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T10:47:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:47:57.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:57 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi169 2024-04-25T10:47:57.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:57 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T10:47:57.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:57 smithi112 ceph-mon[32799]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:47:57.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T10:47:57.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:57 smithi112 ceph-mon[32799]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T10:47:57.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:47:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 -- 172.21.15.19:0/3736132109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54104250 msgr2=0x7f2d54104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3736132109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54104250 0x7f2d54104660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009990 tx=0x7f2d48009ca0 comp rx=0 tx=0).stop 2024-04-25T10:47:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 -- 172.21.15.19:0/3736132109 shutdown_connections 2024-04-25T10:47:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3736132109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d54104bf0 0x7f2d5410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3736132109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d54103120 0x7f2d54103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3736132109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54104250 0x7f2d54104660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 -- 172.21.15.19:0/3736132109 >> 172.21.15.19:0/3736132109 conn(0x7f2d540fe760 msgr2=0x7f2d54100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:47:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 -- 172.21.15.19:0/3736132109 shutdown_connections 2024-04-25T10:47:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.923+0000 7f2d59c15700 1 -- 172.21.15.19:0/3736132109 wait complete. 2024-04-25T10:47:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.924+0000 7f2d59c15700 1 Processor -- start 2024-04-25T10:47:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.924+0000 7f2d59c15700 1 -- start start 2024-04-25T10:47:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.924+0000 7f2d59c15700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54103120 0x7f2d54076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d59c15700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d54104250 0x7f2d5419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d59c15700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d54104bf0 0x7f2d541a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d59c15700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410eb90 con 0x7f2d54103120 2024-04-25T10:47:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d59c15700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410ea10 con 0x7f2d54104250 2024-04-25T10:47:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d59c15700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410ed10 con 0x7f2d54104bf0 2024-04-25T10:47:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54103120 0x7f2d54076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54103120 0x7f2d54076340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33974/0 (socket says 172.21.15.19:33974) 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 -- 172.21.15.19:0/3018673724 learned_addr learned my addr 172.21.15.19:0/3018673724 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d52ffd700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d54104250 0x7f2d5419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d53fff700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d54104bf0 0x7f2d541a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 -- 172.21.15.19:0/3018673724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d54104bf0 msgr2=0x7f2d541a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d54104bf0 0x7f2d541a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 -- 172.21.15.19:0/3018673724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d54104250 msgr2=0x7f2d5419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d54104250 0x7f2d5419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 -- 172.21.15.19:0/3018673724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d541993d0 con 0x7f2d54103120 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d52ffd700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d54104250 0x7f2d5419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.925+0000 7f2d537fe700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54103120 0x7f2d54076340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d950 tx=0x7f2d4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.926+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44017070 con 0x7f2d54103120 2024-04-25T10:47:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.926+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d541996c0 con 0x7f2d54103120 2024-04-25T10:47:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.926+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d54199c50 con 0x7f2d54103120 2024-04-25T10:47:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.926+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d4400f690 con 0x7f2d54103120 2024-04-25T10:47:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.926+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44003aa0 con 0x7f2d54103120 2024-04-25T10:47:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.927+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d54051190 con 0x7f2d54103120 2024-04-25T10:47:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.928+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f2d4400f840 con 0x7f2d54103120 2024-04-25T10:47:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.929+0000 7f2d50ff9700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2d3c0afb70 0x7f2d3c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:47:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.929+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f2d440dd8d0 con 0x7f2d54103120 2024-04-25T10:47:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.929+0000 7f2d52ffd700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2d3c0afb70 0x7f2d3c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:47:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.930+0000 7f2d52ffd700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2d3c0afb70 0x7f2d3c0b2080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d480038a0 tx=0x7f2d4801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:47:57.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:57.934+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d44020030 con 0x7f2d54103120 2024-04-25T10:47:58.026 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48638]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:47:58.026 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48638]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.026 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48638]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.026 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48638]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.027 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48641]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:47:58.027 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48641]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.027 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48641]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.027 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 10:47:57 smithi019 sudo[48641]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:58.161+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2d54109e90 con 0x7f2d3c0afb70 2024-04-25T10:47:58.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42104]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:47:58.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42104]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42104]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42104]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42107]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:47:58.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42107]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42107]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 sudo[42107]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.252 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42098]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:47:58.252 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42098]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.252 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42098]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.253 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42098]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.253 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:47:58.253 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.253 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42101]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.253 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 10:47:57 smithi169 sudo[42101]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41373]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41373]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41373]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41373]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41376]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41376]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41376]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.310 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 10:47:57 smithi112 sudo[41376]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41379]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41379]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41379]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41379]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41382]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41382]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41382]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 sudo[41382]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:47:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48664]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48664]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48667]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:47:58.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48667]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:47:58.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48667]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:47:58.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 sudo[48667]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:47:58.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:47:58.957+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2d440a3570 con 0x7f2d54103120 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:59.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:59.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:59.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:59.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:59.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:47:59.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:47:59.254 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:47:59.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:59 smithi019 ceph-mon[29209]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T10:47:59.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:47:59 smithi019 ceph-mon[29209]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T10:48:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:59 smithi112 ceph-mon[32799]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T10:48:00.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:47:59 smithi112 ceph-mon[32799]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T10:48:00.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:59 smithi169 ceph-mon[33049]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T10:48:00.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:47:59 smithi169 ceph-mon[33049]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T10:48:01.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:00 smithi112 ceph-mon[32799]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:01.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:00 smithi019 ceph-mon[29209]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:01.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:00 smithi169 ceph-mon[33049]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:02.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:01 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3346776531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efc21e6a-f4d3-4abd-aee6-b6a8533142e1"}]: dispatch 2024-04-25T10:48:02.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:01 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3346776531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efc21e6a-f4d3-4abd-aee6-b6a8533142e1"}]': finished 2024-04-25T10:48:02.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:01 smithi112 ceph-mon[32799]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T10:48:02.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:01 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:02.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:01 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1105491563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:02.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:01 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3346776531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efc21e6a-f4d3-4abd-aee6-b6a8533142e1"}]: dispatch 2024-04-25T10:48:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:01 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3346776531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efc21e6a-f4d3-4abd-aee6-b6a8533142e1"}]': finished 2024-04-25T10:48:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:01 smithi019 ceph-mon[29209]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T10:48:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:01 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:01 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1105491563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:02.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:01 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3346776531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efc21e6a-f4d3-4abd-aee6-b6a8533142e1"}]: dispatch 2024-04-25T10:48:02.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:01 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3346776531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efc21e6a-f4d3-4abd-aee6-b6a8533142e1"}]': finished 2024-04-25T10:48:02.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:01 smithi169 ceph-mon[33049]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T10:48:02.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:01 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:02.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:01 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1105491563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:03.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:02 smithi019 ceph-mon[29209]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:03.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:02 smithi169 ceph-mon[33049]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:03.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:02 smithi112 ceph-mon[32799]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:05.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:04 smithi019 ceph-mon[29209]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:05.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:04 smithi169 ceph-mon[33049]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:05.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:04 smithi112 ceph-mon[32799]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:07.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:06 smithi019 ceph-mon[29209]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:07.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:06 smithi169 ceph-mon[33049]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:07.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:06 smithi112 ceph-mon[32799]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:09.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:08 smithi169 ceph-mon[33049]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:09.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T10:48:09.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:09.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:08 smithi169 ceph-mon[33049]: Deploying daemon osd.3 on smithi019 2024-04-25T10:48:09.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:08 smithi112 ceph-mon[32799]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:09.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T10:48:09.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:09.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:08 smithi112 ceph-mon[32799]: Deploying daemon osd.3 on smithi019 2024-04-25T10:48:09.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:08 smithi019 ceph-mon[29209]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:09.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T10:48:09.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:09.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:08 smithi019 ceph-mon[29209]: Deploying daemon osd.3 on smithi019 2024-04-25T10:48:11.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:10 smithi169 ceph-mon[33049]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:11.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:10 smithi112 ceph-mon[32799]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:11.333 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:10 smithi019 ceph-mon[29209]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:13.207 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:12 smithi019 ceph-mon[29209]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:13.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:12 smithi169 ceph-mon[33049]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:13.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:12 smithi112 ceph-mon[32799]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:15.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:15 smithi112 ceph-mon[32799]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:15.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:15 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:15.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:15 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:15.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:15 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:15.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:15 smithi019 ceph-mon[29209]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:15.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:15 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:15.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:15 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:15.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:15 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:15 smithi169 ceph-mon[33049]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:15 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:15.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:15 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:15.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:15 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:17 smithi112 ceph-mon[32799]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:17.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:17.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:17 smithi019 ceph-mon[29209]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:17 smithi169 ceph-mon[33049]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:17.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:17.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:17.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.820+0000 7f2d50ff9700 1 -- 172.21.15.19:0/3018673724 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d54109e90 con 0x7f2d3c0afb70 2024-04-25T10:48:17.823 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-04-25T10:48:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2d3c0afb70 msgr2=0x7f2d3c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2d3c0afb70 0x7f2d3c0b2080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d480038a0 tx=0x7f2d4801a040 comp rx=0 tx=0).stop 2024-04-25T10:48:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54103120 msgr2=0x7f2d54076340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54103120 0x7f2d54076340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d950 tx=0x7f2d4400dc60 comp rx=0 tx=0).stop 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 shutdown_connections 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2d3c0afb70 0x7f2d3c0b2080 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d54104bf0 0x7f2d541a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d54103120 0x7f2d54076340 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 --2- 172.21.15.19:0/3018673724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d54104250 0x7f2d5419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 >> 172.21.15.19:0/3018673724 conn(0x7f2d540fe760 msgr2=0x7f2d54107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 shutdown_connections 2024-04-25T10:48:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:48:17.823+0000 7f2d59c15700 1 -- 172.21.15.19:0/3018673724 wait complete. 2024-04-25T10:48:18.471 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T10:48:18.473 INFO:tasks.cephadm:Deploying osd.4 on smithi112 with /dev/vg_nvme/lv_3... 2024-04-25T10:48:18.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T10:48:18.717 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:48:19.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:18 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:19.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:18 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:19.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:18 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:19.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:18 smithi019 ceph-mon[29209]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:19.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:18 smithi019 ceph-mon[29209]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T10:48:19.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:18 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:19.046 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:18 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:19.046 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:18 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:19.046 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:18 smithi112 ceph-mon[32799]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:19.046 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:18 smithi112 ceph-mon[32799]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T10:48:19.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:18 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:19.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:18 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:19.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:18 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:19.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:18 smithi169 ceph-mon[33049]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:19.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:18 smithi169 ceph-mon[33049]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T10:48:19.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:19 smithi019 ceph-mon[29209]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T10:48:19.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:19 smithi019 ceph-mon[29209]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T10:48:19.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:19 smithi019 ceph-mon[29209]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:48:19.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:19 smithi112 ceph-mon[32799]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T10:48:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:19 smithi112 ceph-mon[32799]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T10:48:20.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:19 smithi112 ceph-mon[32799]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:48:20.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:20.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:19 smithi169 ceph-mon[33049]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T10:48:20.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:19 smithi169 ceph-mon[33049]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T10:48:20.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:19 smithi169 ceph-mon[33049]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:48:20.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:20.656 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T10:48:20.656 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T10:48:20.656 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T10:48:20.656 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T10:48:20.656 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129775 s, 808 MB/s 2024-04-25T10:48:20.656 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.113 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T10:48:20.795+0000 7f1cf2534700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:48:21.113 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 10:48:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T10:48:20.824+0000 7f1cea2e1700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:48:21.222 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:48:21.222 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T10:48:21.223 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.223 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.223 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:21.223 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T10:48:21.223 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.223 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-04-25T10:48:21.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:48:21.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T10:48:21.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:48:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T10:48:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:21.487 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi019 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-25T10:48:22.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:22.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:22.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] boot 2024-04-25T10:48:22.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T10:48:22.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi019 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-25T10:48:22.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.196 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:22.196 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:22.196 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.196 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] boot 2024-04-25T10:48:22.196 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T10:48:22.196 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439]' entity='osd.3' 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi019 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:22.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:22.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:22.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: osd.3 [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] boot 2024-04-25T10:48:22.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T10:48:22.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:48:23.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 -- 172.21.15.112:0/3413497023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830104ce0 msgr2=0x7fc830107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:23.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 --2- 172.21.15.112:0/3413497023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830104ce0 0x7fc830107100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc820009990 tx=0x7fc820009ca0 comp rx=0 tx=0).stop 2024-04-25T10:48:23.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 -- 172.21.15.112:0/3413497023 shutdown_connections 2024-04-25T10:48:23.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 --2- 172.21.15.112:0/3413497023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc830109fa0 0x7fc83010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 --2- 172.21.15.112:0/3413497023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830104ce0 0x7fc830107100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 --2- 172.21.15.112:0/3413497023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc830107640 0x7fc830109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 -- 172.21.15.112:0/3413497023 >> 172.21.15.112:0/3413497023 conn(0x7fc8300fe740 msgr2=0x7fc830100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 -- 172.21.15.112:0/3413497023 shutdown_connections 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.016+0000 7fc8355b0700 1 -- 172.21.15.112:0/3413497023 wait complete. 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 Processor -- start 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 -- start start 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc830104ce0 0x7fc83019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830107640 0x7fc83019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc830109fa0 0x7fc8301a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc83010fc20 con 0x7fc830104ce0 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc83010faa0 con 0x7fc830107640 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.017+0000 7fc8355b0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc83010fda0 con 0x7fc830109fa0 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830107640 0x7fc83019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:23.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830107640 0x7fc83019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39964/0 (socket says 172.21.15.112:39964) 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc82effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc830104ce0 0x7fc83019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 -- 172.21.15.112:0/1687957783 learned_addr learned my addr 172.21.15.112:0/1687957783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc82f7fe700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc830109fa0 0x7fc8301a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 -- 172.21.15.112:0/1687957783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc830109fa0 msgr2=0x7fc8301a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc830109fa0 0x7fc8301a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 -- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc830104ce0 msgr2=0x7fc83019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc830104ce0 0x7fc83019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 -- 172.21.15.112:0/1687957783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8301a22c0 con 0x7fc830107640 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc82effd700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc830104ce0 0x7fc83019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8267fc700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830107640 0x7fc83019aca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc81800d950 tx=0x7fc81800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc82cff9700 1 -- 172.21.15.112:0/1687957783 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818016070 con 0x7fc830107640 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8301a25b0 con 0x7fc830107640 2024-04-25T10:48:23.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc82cff9700 1 -- 172.21.15.112:0/1687957783 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81800fd40 con 0x7fc830107640 2024-04-25T10:48:23.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc82cff9700 1 -- 172.21.15.112:0/1687957783 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81800e3b0 con 0x7fc830107640 2024-04-25T10:48:23.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.018+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8301a2b40 con 0x7fc830107640 2024-04-25T10:48:23.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.019+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc830051190 con 0x7fc830107640 2024-04-25T10:48:23.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.021+0000 7fc82cff9700 1 -- 172.21.15.112:0/1687957783 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc818019070 con 0x7fc830107640 2024-04-25T10:48:23.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.021+0000 7fc82cff9700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc81c0afb80 0x7fc81c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:23.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.021+0000 7fc82effd700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc81c0afb80 0x7fc81c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:23.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.021+0000 7fc82cff9700 1 -- 172.21.15.112:0/1687957783 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fc8180dd520 con 0x7fc830107640 2024-04-25T10:48:23.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.022+0000 7fc82effd700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc81c0afb80 0x7fc81c0b2090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc820006010 tx=0x7fc82000b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:48:23.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.025+0000 7fc82cff9700 1 -- 172.21.15.112:0/1687957783 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8180a7280 con 0x7fc830107640 2024-04-25T10:48:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:23.229+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc830063ad0 con 0x7fc81c0afb80 2024-04-25T10:48:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:23 smithi112 ceph-mon[32799]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:23 smithi112 ceph-mon[32799]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T10:48:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:48:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:48:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:23.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:23 smithi169 ceph-mon[33049]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:23.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:23 smithi169 ceph-mon[33049]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T10:48:23.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:48:23.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:48:23.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:23.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:23 smithi019 ceph-mon[29209]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:23.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:23 smithi019 ceph-mon[29209]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T10:48:23.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:48:23.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:48:23.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:24.719 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:24 smithi112 ceph-mon[32799]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:48:24.720 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:24 smithi112 ceph-mon[32799]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T10:48:24.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:24 smithi169 ceph-mon[33049]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:48:24.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:24 smithi169 ceph-mon[33049]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T10:48:24.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:24 smithi019 ceph-mon[29209]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:48:24.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:24 smithi019 ceph-mon[29209]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T10:48:25.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:25 smithi169 ceph-mon[33049]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:25 smithi112 ceph-mon[32799]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:25 smithi019 ceph-mon[29209]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:26.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:26 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/3593757285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]: dispatch 2024-04-25T10:48:26.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:26 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]: dispatch 2024-04-25T10:48:26.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:26 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]': finished 2024-04-25T10:48:26.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:26 smithi112 ceph-mon[32799]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T10:48:26.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:26 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:26.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:26 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/3593757285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]: dispatch 2024-04-25T10:48:26.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:26 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]: dispatch 2024-04-25T10:48:26.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:26 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]': finished 2024-04-25T10:48:26.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:26 smithi169 ceph-mon[33049]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T10:48:26.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:26 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:26.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:26 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/3593757285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]: dispatch 2024-04-25T10:48:26.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:26 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]: dispatch 2024-04-25T10:48:26.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:26 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1ec5ae4-0515-4c9d-a701-7ea0e010534f"}]': finished 2024-04-25T10:48:26.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:26 smithi019 ceph-mon[29209]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T10:48:26.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:26 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:27.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:27 smithi169 ceph-mon[33049]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:27.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:27 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/4196448442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:27.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:27 smithi112 ceph-mon[32799]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:27.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:27 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/4196448442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:27.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:27 smithi019 ceph-mon[29209]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:27.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:27 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/4196448442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:29.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:29 smithi169 ceph-mon[33049]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:48:29.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:29 smithi112 ceph-mon[32799]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:48:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:29 smithi019 ceph-mon[29209]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:48:31.489 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:31 smithi112 ceph-mon[32799]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T10:48:31.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:31 smithi169 ceph-mon[33049]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T10:48:31.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:31 smithi019 ceph-mon[29209]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T10:48:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:33 smithi112 ceph-mon[32799]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T10:48:33.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:33 smithi169 ceph-mon[33049]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T10:48:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:33 smithi019 ceph-mon[29209]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T10:48:34.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:34 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T10:48:34.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:34 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:34.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:34 smithi112 ceph-mon[32799]: Deploying daemon osd.4 on smithi112 2024-04-25T10:48:34.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:34 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T10:48:34.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:34 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:34.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:34 smithi169 ceph-mon[33049]: Deploying daemon osd.4 on smithi112 2024-04-25T10:48:34.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:34 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T10:48:34.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:34 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:34.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:34 smithi019 ceph-mon[29209]: Deploying daemon osd.4 on smithi112 2024-04-25T10:48:35.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:35 smithi112 ceph-mon[32799]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:48:35.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:35 smithi169 ceph-mon[33049]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:48:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:35 smithi019 ceph-mon[29209]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:48:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:37 smithi112 ceph-mon[32799]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:48:37.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:37 smithi169 ceph-mon[33049]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:48:37.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:37 smithi019 ceph-mon[29209]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:48:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:39 smithi112 ceph-mon[32799]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:48:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:39.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:39.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:39 smithi169 ceph-mon[33049]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:48:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:39.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:39.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:39 smithi019 ceph-mon[29209]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:48:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:39.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:39.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:41.735 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:41 smithi112 ceph-mon[32799]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:41 smithi169 ceph-mon[33049]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:41 smithi019 ceph-mon[29209]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:42.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.245+0000 7fc82cff9700 1 -- 172.21.15.112:0/1687957783 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc830063ad0 con 0x7fc81c0afb80 2024-04-25T10:48:42.247 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 4 on host 'smithi112' 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc81c0afb80 msgr2=0x7fc81c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc81c0afb80 0x7fc81c0b2090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc820006010 tx=0x7fc82000b720 comp rx=0 tx=0).stop 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830107640 msgr2=0x7fc83019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830107640 0x7fc83019aca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc81800d950 tx=0x7fc81800dc60 comp rx=0 tx=0).stop 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 shutdown_connections 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc81c0afb80 0x7fc81c0b2090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc830109fa0 0x7fc8301a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc830104ce0 0x7fc83019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 --2- 172.21.15.112:0/1687957783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc830107640 0x7fc83019aca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 >> 172.21.15.112:0/1687957783 conn(0x7fc8300fe740 msgr2=0x7fc830105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:48:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 shutdown_connections 2024-04-25T10:48:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:48:42.247+0000 7fc8355b0700 1 -- 172.21.15.112:0/1687957783 wait complete. 2024-04-25T10:48:42.891 DEBUG:teuthology.orchestra.run.smithi112:osd.4> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T10:48:42.894 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_3... 2024-04-25T10:48:42.894 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T10:48:43.142 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:48:43.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:43 smithi112 ceph-mon[32799]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:43.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:43 smithi112 ceph-mon[32799]: from='osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:48:43.377 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:43 smithi112 ceph-mon[32799]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:48:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:43 smithi169 ceph-mon[33049]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:43 smithi169 ceph-mon[33049]: from='osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:48:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:43 smithi169 ceph-mon[33049]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:48:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:43 smithi019 ceph-mon[29209]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:43.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:43 smithi019 ceph-mon[29209]: from='osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:48:43.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:43 smithi019 ceph-mon[29209]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:48:44.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:44 smithi169 ceph-mon[33049]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T10:48:44.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:44 smithi169 ceph-mon[33049]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T10:48:44.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:44 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:44.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:44 smithi169 ceph-mon[33049]: from='osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:48:44.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:44 smithi169 ceph-mon[33049]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:48:44.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:44 smithi112 ceph-mon[32799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T10:48:44.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:44 smithi112 ceph-mon[32799]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T10:48:44.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:44 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:44.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:44 smithi112 ceph-mon[32799]: from='osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:48:44.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:44 smithi112 ceph-mon[32799]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:48:44.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:44 smithi019 ceph-mon[29209]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T10:48:44.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:44 smithi019 ceph-mon[29209]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T10:48:44.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:44 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:44.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:44 smithi019 ceph-mon[29209]: from='osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:48:44.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:44 smithi019 ceph-mon[29209]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:48:45.001 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T10:48:45.001 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T10:48:45.001 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T10:48:45.001 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T10:48:45.001 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0181605 s, 577 MB/s 2024-04-25T10:48:45.002 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T10:48:45.575 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:45 smithi169 ceph-mon[33049]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:45.575 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:45 smithi169 ceph-mon[33049]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:48:45.575 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:45 smithi169 ceph-mon[33049]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T10:48:45.575 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:45.575 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:45.575 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:45 smithi169 ceph-mon[33049]: from='osd.4 ' entity='osd.4' 2024-04-25T10:48:45.603 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-04-25T10:48:45.644 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-mon[32799]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:45.644 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-mon[32799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:48:45.644 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-mon[32799]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T10:48:45.644 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:45.644 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:45.644 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-mon[32799]: from='osd.4 ' entity='osd.4' 2024-04-25T10:48:45.644 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4[44184]: 2024-04-25T10:48:45.332+0000 7fb437a9c700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:48:45.644 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 10:48:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4[44184]: 2024-04-25T10:48:45.466+0000 7fb42f048700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:48:45.848 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:48:45.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:45 smithi019 ceph-mon[29209]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:45.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:45 smithi019 ceph-mon[29209]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:48:45.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:45 smithi019 ceph-mon[29209]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T10:48:45.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:45.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:45.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:45 smithi019 ceph-mon[29209]: from='osd.4 ' entity='osd.4' 2024-04-25T10:48:46.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:46.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:46.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:46.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:48:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:48:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:48:46.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:48:46.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 -- 172.21.15.169:0/2706170847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4102b70 msgr2=0x7fccd410c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 --2- 172.21.15.169:0/2706170847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4102b70 0x7fccd410c790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00b0a0 tx=0x7fcccc00b3b0 comp rx=0 tx=0).stop 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 -- 172.21.15.169:0/2706170847 shutdown_connections 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 --2- 172.21.15.169:0/2706170847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4102b70 0x7fccd410c790 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 --2- 172.21.15.169:0/2706170847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd41057d0 0x7fccd4105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 --2- 172.21.15.169:0/2706170847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd4102220 0x7fccd4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 -- 172.21.15.169:0/2706170847 >> 172.21.15.169:0/2706170847 conn(0x7fccd40fded0 msgr2=0x7fccd41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 -- 172.21.15.169:0/2706170847 shutdown_connections 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.423+0000 7fccdb234700 1 -- 172.21.15.169:0/2706170847 wait complete. 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 Processor -- start 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 -- start start 2024-04-25T10:48:47.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102220 0x7fccd419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:47.426 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd4102b70 0x7fccd419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:47.427 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd41057d0 0x7fccd41a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410fc90 con 0x7fccd4102b70 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410fb10 con 0x7fccd4102220 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccdb234700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410fe10 con 0x7fccd41057d0 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccd97d1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd41057d0 0x7fccd41a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccd97d1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd41057d0 0x7fccd41a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58572/0 (socket says 172.21.15.169:58572) 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccd97d1700 1 -- 172.21.15.169:0/3163251395 learned_addr learned my addr 172.21.15.169:0/3163251395 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccd8fd0700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102220 0x7fccd419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.424+0000 7fccd3fff700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd4102b70 0x7fccd419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd97d1700 1 -- 172.21.15.169:0/3163251395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102220 msgr2=0x7fccd419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd97d1700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102220 0x7fccd419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:47.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd97d1700 1 -- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd4102b70 msgr2=0x7fccd419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd97d1700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd4102b70 0x7fccd419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd97d1700 1 -- 172.21.15.169:0/3163251395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd41a2300 con 0x7fccd41057d0 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd8fd0700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102220 0x7fccd419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd3fff700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd4102b70 0x7fccd419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd97d1700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd41057d0 0x7fccd41a1c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcccc006130 tx=0x7fcccc00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd1ffb700 1 -- 172.21.15.169:0/3163251395 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc021070 con 0x7fccd41057d0 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd1ffb700 1 -- 172.21.15.169:0/3163251395 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcccc01d450 con 0x7fccd41057d0 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd41a2590 con 0x7fccd41057d0 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccd1ffb700 1 -- 172.21.15.169:0/3163251395 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc01b680 con 0x7fccd41057d0 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.425+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41a2ad0 con 0x7fccd41057d0 2024-04-25T10:48:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.426+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd4051190 con 0x7fccd41057d0 2024-04-25T10:48:47.430 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.428+0000 7fccd1ffb700 1 -- 172.21.15.169:0/3163251395 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fcccc024070 con 0x7fccd41057d0 2024-04-25T10:48:47.430 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.429+0000 7fccd1ffb700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fccbc0afb80 0x7fccbc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:48:47.430 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.429+0000 7fccd1ffb700 1 -- 172.21.15.169:0/3163251395 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fcccc00f020 con 0x7fccd41057d0 2024-04-25T10:48:47.430 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.429+0000 7fccd8fd0700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fccbc0afb80 0x7fccbc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:48:47.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.430+0000 7fccd8fd0700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fccbc0afb80 0x7fccbc0b2090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fccd406bf10 tx=0x7fccc4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:48:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.433+0000 7fccd1ffb700 1 -- 172.21.15.169:0/3163251395 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcccc02b020 con 0x7fccd41057d0 2024-04-25T10:48:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:47 smithi169 ceph-mon[33049]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:47.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:47 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi112 2024-04-25T10:48:47.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:47 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T10:48:47.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:47 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:47.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:47 smithi169 ceph-mon[33049]: osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] boot 2024-04-25T10:48:47.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:47 smithi169 ceph-mon[33049]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T10:48:47.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:47 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:47.640 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:48:47.639+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fccd4063ad0 con 0x7fccbc0afb80 2024-04-25T10:48:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:47 smithi112 ceph-mon[32799]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:47 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi112 2024-04-25T10:48:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:47 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T10:48:47.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:47 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:47.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:47 smithi112 ceph-mon[32799]: osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] boot 2024-04-25T10:48:47.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:47 smithi112 ceph-mon[32799]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T10:48:47.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:47 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:47.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:47 smithi019 ceph-mon[29209]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:48:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:47 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi112 2024-04-25T10:48:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:47 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T10:48:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:47 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:47 smithi019 ceph-mon[29209]: osd.4 [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] boot 2024-04-25T10:48:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:47 smithi019 ceph-mon[29209]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T10:48:47.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:47 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:48:48.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:48 smithi169 ceph-mon[33049]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T10:48:48.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:48 smithi169 ceph-mon[33049]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:48:48.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:48 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:48:48.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:48 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:48:48.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:48 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:48.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:48 smithi112 ceph-mon[32799]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T10:48:48.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:48 smithi112 ceph-mon[32799]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:48:48.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:48 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:48:48.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:48 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:48:48.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:48 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:48.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:48 smithi019 ceph-mon[29209]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T10:48:48.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:48 smithi019 ceph-mon[29209]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:48:48.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:48 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:48:48.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:48 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:48:48.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:48 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:49.552 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:49 smithi169 ceph-mon[33049]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:49.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:49 smithi112 ceph-mon[32799]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:49.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:49 smithi019 ceph-mon[29209]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:50.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:50 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/911312611' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]: dispatch 2024-04-25T10:48:50.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:50 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]: dispatch 2024-04-25T10:48:50.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:50 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]': finished 2024-04-25T10:48:50.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:50 smithi169 ceph-mon[33049]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T10:48:50.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:50 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:48:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:50 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/911312611' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]: dispatch 2024-04-25T10:48:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:50 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]: dispatch 2024-04-25T10:48:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:50 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]': finished 2024-04-25T10:48:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:50 smithi112 ceph-mon[32799]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T10:48:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:50 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:48:50.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:50 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/911312611' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]: dispatch 2024-04-25T10:48:50.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:50 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]: dispatch 2024-04-25T10:48:50.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:50 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c271d7f-d82b-4849-a765-f0d0de0ad3db"}]': finished 2024-04-25T10:48:50.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:50 smithi019 ceph-mon[29209]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T10:48:50.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:50 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:48:51.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:51 smithi169 ceph-mon[33049]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:51.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:51 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/1236877763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:51.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:51 smithi019 ceph-mon[29209]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:51.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:51 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/1236877763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:51.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:51 smithi112 ceph-mon[32799]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:51.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:51 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/1236877763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:48:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:53 smithi169 ceph-mon[33049]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:53.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:53 smithi112 ceph-mon[32799]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:53.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:53 smithi019 ceph-mon[29209]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:55.732 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:55 smithi169 ceph-mon[33049]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:55 smithi112 ceph-mon[32799]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:55.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:55 smithi019 ceph-mon[29209]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:57.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:57 smithi169 ceph-mon[33049]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:57.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:57 smithi112 ceph-mon[32799]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:57.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:57 smithi019 ceph-mon[29209]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:58.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T10:48:58.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:58.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T10:48:58.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:58.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T10:48:58.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:48:59.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:59 smithi169 ceph-mon[33049]: Deploying daemon osd.5 on smithi169 2024-04-25T10:48:59.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:48:59 smithi169 ceph-mon[33049]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:59.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:59 smithi112 ceph-mon[32799]: Deploying daemon osd.5 on smithi169 2024-04-25T10:48:59.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:48:59 smithi112 ceph-mon[32799]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:48:59.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:59 smithi019 ceph-mon[29209]: Deploying daemon osd.5 on smithi169 2024-04-25T10:48:59.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:48:59 smithi019 ceph-mon[29209]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:01.730 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:01 smithi169 ceph-mon[33049]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:01.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:01 smithi112 ceph-mon[32799]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:01.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:01 smithi019 ceph-mon[29209]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:03.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:03 smithi169 ceph-mon[33049]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:03.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:03 smithi112 ceph-mon[32799]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:03.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:03 smithi019 ceph-mon[29209]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:04.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:04 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:04.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:04 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:04.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:04 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:05.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:04 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:05.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:04 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:05.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:04 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:05.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:04 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:05.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:04 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:05.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:04 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:05 smithi169 ceph-mon[33049]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:06.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:05 smithi112 ceph-mon[32799]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:06.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:05 smithi019 ceph-mon[29209]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:07.012 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-04-25T10:49:07.013 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.010+0000 7fccd1ffb700 1 -- 172.21.15.169:0/3163251395 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fccd4063ad0 con 0x7fccbc0afb80 2024-04-25T10:49:07.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fccbc0afb80 msgr2=0x7fccbc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:07.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fccbc0afb80 0x7fccbc0b2090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fccd406bf10 tx=0x7fccc4009260 comp rx=0 tx=0).stop 2024-04-25T10:49:07.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd41057d0 msgr2=0x7fccd41a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:07.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd41057d0 0x7fccd41a1c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcccc006130 tx=0x7fcccc00be60 comp rx=0 tx=0).stop 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 shutdown_connections 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fccbc0afb80 0x7fccbc0b2090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd41057d0 0x7fccd41a1c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102220 0x7fccd419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 --2- 172.21.15.169:0/3163251395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd4102b70 0x7fccd419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 >> 172.21.15.169:0/3163251395 conn(0x7fccd40fded0 msgr2=0x7fccd4108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 shutdown_connections 2024-04-25T10:49:07.016 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:49:07.013+0000 7fccdb234700 1 -- 172.21.15.169:0/3163251395 wait complete. 2024-04-25T10:49:07.730 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T10:49:07.733 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-04-25T10:49:07.733 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T10:49:07.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:07 smithi169 ceph-mon[33049]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:07.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:07.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:07.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:07.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:07.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:07 smithi169 ceph-mon[33049]: from='osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:49:07.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:07 smithi169 ceph-mon[33049]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:49:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:49:08.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:07 smithi112 ceph-mon[32799]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:08.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:07 smithi112 ceph-mon[32799]: from='osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:49:08.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:07 smithi112 ceph-mon[32799]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:49:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:07 smithi019 ceph-mon[29209]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:07 smithi019 ceph-mon[29209]: from='osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:49:08.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:07 smithi019 ceph-mon[29209]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:49:09.383 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-mon[33049]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:09.383 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-mon[33049]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T10:49:09.383 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-mon[33049]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T10:49:09.383 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:09.384 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-mon[33049]: from='osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:49:09.384 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-mon[33049]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:49:09.384 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T10:49:09.373+0000 7fe960bcd700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:49:09.384 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 10:49:09 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T10:49:09.381+0000 7fe95c982700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:49:09.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:09 smithi019 ceph-mon[29209]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:09.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:09 smithi019 ceph-mon[29209]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T10:49:09.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:09 smithi019 ceph-mon[29209]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T10:49:09.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:09 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:09.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:09 smithi019 ceph-mon[29209]: from='osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:49:09.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:09 smithi019 ceph-mon[29209]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:49:09.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:09 smithi112 ceph-mon[32799]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:09.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:09 smithi112 ceph-mon[32799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T10:49:09.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:09 smithi112 ceph-mon[32799]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T10:49:09.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:09 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:09.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:09 smithi112 ceph-mon[32799]: from='osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:49:09.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:09 smithi112 ceph-mon[32799]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:49:10.109 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T10:49:10.109 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T10:49:10.109 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T10:49:10.109 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T10:49:10.109 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130604 s, 803 MB/s 2024-04-25T10:49:10.109 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T10:49:10.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:10 smithi169 ceph-mon[33049]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:49:10.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:10 smithi169 ceph-mon[33049]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T10:49:10.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:10 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:10.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:10 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:10.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:10 smithi019 ceph-mon[29209]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:49:10.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:10 smithi019 ceph-mon[29209]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T10:49:10.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:10 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:10.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:10 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:10.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-04-25T10:49:10.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:10 smithi112 ceph-mon[32799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:49:10.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:10 smithi112 ceph-mon[32799]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T10:49:10.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:10 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:10.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:10 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] boot 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:11.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi169 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:11.712 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] boot 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi169 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.745 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:11.746 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: osd.5 [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] boot 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:49:11.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi169 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:11.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 -- 172.21.15.19:0/923869520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508104d20 msgr2=0x7f0508107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 --2- 172.21.15.19:0/923869520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508104d20 0x7f0508107140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009990 tx=0x7f04f8009ca0 comp rx=0 tx=0).stop 2024-04-25T10:49:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 -- 172.21.15.19:0/923869520 shutdown_connections 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 --2- 172.21.15.19:0/923869520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508109fe0 0x7f050810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 --2- 172.21.15.19:0/923869520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508104d20 0x7f0508107140 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 --2- 172.21.15.19:0/923869520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0508107680 0x7f0508109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 -- 172.21.15.19:0/923869520 >> 172.21.15.19:0/923869520 conn(0x7f05080fe760 msgr2=0x7f0508100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 -- 172.21.15.19:0/923869520 shutdown_connections 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.546+0000 7f050e59b700 1 -- 172.21.15.19:0/923869520 wait complete. 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.547+0000 7f050e59b700 1 Processor -- start 2024-04-25T10:49:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.547+0000 7f050e59b700 1 -- start start 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050e59b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508104d20 0x7f050819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050e59b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0508107680 0x7f050819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050e59b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508109fe0 0x7f05081a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050e59b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f050810fcd0 con 0x7f0508109fe0 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050e59b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f050810fb50 con 0x7f0508104d20 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050e59b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f050810fe50 con 0x7f0508107680 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508109fe0 0x7f05081a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508109fe0 0x7f05081a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38230/0 (socket says 172.21.15.19:38230) 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 -- 172.21.15.19:0/2129639909 learned_addr learned my addr 172.21.15.19:0/2129639909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f0507fff700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508104d20 0x7f050819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f05077fe700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0508107680 0x7f050819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 -- 172.21.15.19:0/2129639909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0508107680 msgr2=0x7f050819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0508107680 0x7f050819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 -- 172.21.15.19:0/2129639909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508104d20 msgr2=0x7f050819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508104d20 0x7f050819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 -- 172.21.15.19:0/2129639909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05081a2380 con 0x7f0508109fe0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.548+0000 7f050cb38700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508109fe0 0x7f05081a1d00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0500007ee0 tx=0x7f050000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.549+0000 7f05057fa700 1 -- 172.21.15.19:0/2129639909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050000dbd0 con 0x7f0508109fe0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.549+0000 7f05057fa700 1 -- 172.21.15.19:0/2129639909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050000dd70 con 0x7f0508109fe0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.549+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05081a2670 con 0x7f0508109fe0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.549+0000 7f05057fa700 1 -- 172.21.15.19:0/2129639909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500005990 con 0x7f0508109fe0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.549+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a2be0 con 0x7f0508109fe0 2024-04-25T10:49:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.550+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0508051190 con 0x7f0508109fe0 2024-04-25T10:49:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.551+0000 7f05057fa700 1 -- 172.21.15.19:0/2129639909 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f050001a020 con 0x7f0508109fe0 2024-04-25T10:49:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.552+0000 7f05057fa700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f04f40afb80 0x7f04f40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.552+0000 7f0507fff700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f04f40afb80 0x7f04f40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.552+0000 7f05057fa700 1 -- 172.21.15.19:0/2129639909 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f05000ddc00 con 0x7f0508109fe0 2024-04-25T10:49:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.553+0000 7f0507fff700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f04f40afb80 0x7f04f40b2090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b5c0 tx=0x7f04f801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:49:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.556+0000 7f05057fa700 1 -- 172.21.15.19:0/2129639909 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0500010070 con 0x7f0508109fe0 2024-04-25T10:49:12.735 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:12 smithi112 ceph-mon[32799]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T10:49:12.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:12 smithi169 ceph-mon[33049]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T10:49:12.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:12.776+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0508063ad0 con 0x7f04f40afb80 2024-04-25T10:49:12.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:12 smithi019 ceph-mon[29209]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T10:49:13.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:13 smithi019 ceph-mon[29209]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:13.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:13 smithi019 ceph-mon[29209]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T10:49:13.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:49:13.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:49:13.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:13 smithi169 ceph-mon[33049]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:13 smithi169 ceph-mon[33049]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T10:49:13.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:49:13.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:49:13.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:13.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:13 smithi112 ceph-mon[32799]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:13.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:13 smithi112 ceph-mon[32799]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T10:49:13.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:49:13.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:49:13.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:14.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:14 smithi019 ceph-mon[29209]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:49:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:14 smithi169 ceph-mon[33049]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:49:14.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:14 smithi112 ceph-mon[32799]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:49:15.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:15 smithi169 ceph-mon[33049]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:15.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:15 smithi112 ceph-mon[32799]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:15.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:15 smithi019 ceph-mon[29209]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:16.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:16 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1600706744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a24aa3b-f969-467b-8fdf-f0550d37a071"}]: dispatch 2024-04-25T10:49:16.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:16 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1600706744' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a24aa3b-f969-467b-8fdf-f0550d37a071"}]': finished 2024-04-25T10:49:16.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:16 smithi169 ceph-mon[33049]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T10:49:16.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:16.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:16 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2217670938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:49:16.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:16 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1600706744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a24aa3b-f969-467b-8fdf-f0550d37a071"}]: dispatch 2024-04-25T10:49:16.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:16 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1600706744' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a24aa3b-f969-467b-8fdf-f0550d37a071"}]': finished 2024-04-25T10:49:16.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:16 smithi112 ceph-mon[32799]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T10:49:16.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:16.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:16 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2217670938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:49:16.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:16 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1600706744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a24aa3b-f969-467b-8fdf-f0550d37a071"}]: dispatch 2024-04-25T10:49:16.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:16 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1600706744' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a24aa3b-f969-467b-8fdf-f0550d37a071"}]': finished 2024-04-25T10:49:16.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:16 smithi019 ceph-mon[29209]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T10:49:16.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:16.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:16 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2217670938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:49:17.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:17 smithi112 ceph-mon[32799]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:17.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:17 smithi019 ceph-mon[29209]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:18.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:17 smithi169 ceph-mon[33049]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:19.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:19 smithi112 ceph-mon[32799]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:49:19.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:19 smithi019 ceph-mon[29209]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:49:20.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:19 smithi169 ceph-mon[33049]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:49:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:21 smithi112 ceph-mon[32799]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:49:21.822 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:21 smithi169 ceph-mon[33049]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:49:21.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:21 smithi019 ceph-mon[29209]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:49:23.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:23 smithi019 ceph-mon[29209]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:49:23.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:23 smithi112 ceph-mon[32799]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:49:24.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:23 smithi169 ceph-mon[33049]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:49:24.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T10:49:24.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:24.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:24 smithi019 ceph-mon[29209]: Deploying daemon osd.6 on smithi019 2024-04-25T10:49:24.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T10:49:24.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:24.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:24 smithi112 ceph-mon[32799]: Deploying daemon osd.6 on smithi019 2024-04-25T10:49:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T10:49:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:25.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:24 smithi169 ceph-mon[33049]: Deploying daemon osd.6 on smithi019 2024-04-25T10:49:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:25 smithi112 ceph-mon[32799]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:49:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:25 smithi019 ceph-mon[29209]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:49:26.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:25 smithi169 ceph-mon[33049]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:49:27.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:27 smithi019 ceph-mon[29209]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T10:49:27.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:27 smithi112 ceph-mon[32799]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T10:49:28.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:27 smithi169 ceph-mon[33049]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T10:49:29.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:29 smithi112 ceph-mon[32799]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:49:29.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:29.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:29 smithi019 ceph-mon[29209]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:49:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:29 smithi169 ceph-mon[33049]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:49:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:30.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:30.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:31.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:31.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:31 smithi112 ceph-mon[32799]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:31.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:31 smithi019 ceph-mon[29209]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:32.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:31 smithi169 ceph-mon[33049]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:33.147 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-04-25T10:49:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.145+0000 7f05057fa700 1 -- 172.21.15.19:0/2129639909 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0508063ad0 con 0x7f04f40afb80 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f04f40afb80 msgr2=0x7f04f40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f04f40afb80 0x7f04f40b2090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b5c0 tx=0x7f04f801a040 comp rx=0 tx=0).stop 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508109fe0 msgr2=0x7f05081a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508109fe0 0x7f05081a1d00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0500007ee0 tx=0x7f050000d3b0 comp rx=0 tx=0).stop 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 shutdown_connections 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f04f40afb80 0x7f04f40b2090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0508109fe0 0x7f05081a1d00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508104d20 0x7f050819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 --2- 172.21.15.19:0/2129639909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0508107680 0x7f050819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 >> 172.21.15.19:0/2129639909 conn(0x7f05080fe760 msgr2=0x7f05081057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:49:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 shutdown_connections 2024-04-25T10:49:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:49:33.148+0000 7f050e59b700 1 -- 172.21.15.19:0/2129639909 wait complete. 2024-04-25T10:49:33.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:33 smithi019 ceph-mon[29209]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:33.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:33 smithi019 ceph-mon[29209]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T10:49:33.812 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T10:49:33.815 INFO:tasks.cephadm:Deploying osd.7 on smithi112 with /dev/vg_nvme/lv_2... 2024-04-25T10:49:33.815 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T10:49:33.841 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:33 smithi112 ceph-mon[32799]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:33.842 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.842 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.842 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.842 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:33.842 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:33 smithi112 ceph-mon[32799]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T10:49:34.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:33 smithi169 ceph-mon[33049]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:34.032 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:34.032 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:34.032 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:34.033 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:34.033 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:33 smithi169 ceph-mon[33049]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T10:49:34.060 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:49:35.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:35 smithi169 ceph-mon[33049]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:35.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:35 smithi169 ceph-mon[33049]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T10:49:35.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:35 smithi169 ceph-mon[33049]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T10:49:35.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:35 smithi169 ceph-mon[33049]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:49:35.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:35 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:35.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:35 smithi112 ceph-mon[32799]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:35.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:35 smithi112 ceph-mon[32799]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T10:49:35.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:35 smithi112 ceph-mon[32799]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T10:49:35.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:35 smithi112 ceph-mon[32799]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:49:35.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:35 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:35 smithi019 ceph-mon[29209]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:35 smithi019 ceph-mon[29209]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T10:49:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:35 smithi019 ceph-mon[29209]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T10:49:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:35 smithi019 ceph-mon[29209]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:49:35.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:35 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:35.994 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T10:49:35.994 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T10:49:35.994 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T10:49:35.994 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T10:49:35.994 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114587 s, 915 MB/s 2024-04-25T10:49:35.994 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T10:49:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:36 smithi112 ceph-mon[32799]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:49:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:36 smithi112 ceph-mon[32799]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T10:49:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:36 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:36.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:36 smithi112 ceph-mon[32799]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' 2024-04-25T10:49:36.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-04-25T10:49:36.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:36 smithi169 ceph-mon[33049]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:49:36.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:36 smithi169 ceph-mon[33049]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T10:49:36.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:36.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:36 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:36.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:36 smithi169 ceph-mon[33049]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' 2024-04-25T10:49:36.781 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:36 smithi019 ceph-mon[29209]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:49:36.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:36 smithi019 ceph-mon[29209]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T10:49:36.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:36.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:36 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:36.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:36 smithi019 ceph-mon[29209]: from='osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193]' entity='osd.6' 2024-04-25T10:49:36.782 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 10:49:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T10:49:36.391+0000 7fa52fd81700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:49:36.782 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 10:49:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T10:49:36.435+0000 7fa527b2e700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:49:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:49:37.746 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:37.747 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] boot 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T10:49:37.748 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:37 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:37.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:49:37.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:49:37.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:37.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] boot 2024-04-25T10:49:37.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T10:49:37.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:37 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:49:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:49:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:49:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: osd.6 [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] boot 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T10:49:37.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:37 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:49:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 -- 172.21.15.112:0/1371212344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84441033b0 msgr2=0x7f84441037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 --2- 172.21.15.112:0/1371212344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84441033b0 0x7f84441037c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f842c0099c0 tx=0x7f842c009cd0 comp rx=0 tx=0).stop 2024-04-25T10:49:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 -- 172.21.15.112:0/1371212344 shutdown_connections 2024-04-25T10:49:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 --2- 172.21.15.112:0/1371212344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8444103d00 0x7f844410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 --2- 172.21.15.112:0/1371212344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8444104690 0x7f8444104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 --2- 172.21.15.112:0/1371212344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84441033b0 0x7f84441037c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 -- 172.21.15.112:0/1371212344 >> 172.21.15.112:0/1371212344 conn(0x7f84440fdef0 msgr2=0x7f84441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 -- 172.21.15.112:0/1371212344 shutdown_connections 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.572+0000 7f844a030700 1 -- 172.21.15.112:0/1371212344 wait complete. 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 Processor -- start 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 -- start start 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84441033b0 0x7f8444199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8444103d00 0x7f8444199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8444104690 0x7f84441a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f844410ec20 con 0x7f84441033b0 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f844410eaa0 con 0x7f8444104690 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f844a030700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f844410eda0 con 0x7f8444103d00 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f8443fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8444104690 0x7f84441a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f8443fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8444104690 0x7f84441a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60916/0 (socket says 172.21.15.112:60916) 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f8443fff700 1 -- 172.21.15.112:0/494611481 learned_addr learned my addr 172.21.15.112:0/494611481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.573+0000 7f84437fe700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84441033b0 0x7f8444199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f843bfff700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8444103d00 0x7f8444199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f8443fff700 1 -- 172.21.15.112:0/494611481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8444103d00 msgr2=0x7f8444199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f8443fff700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8444103d00 0x7f8444199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f8443fff700 1 -- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84441033b0 msgr2=0x7f8444199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f8443fff700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84441033b0 0x7f8444199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f8443fff700 1 -- 172.21.15.112:0/494611481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441a1260 con 0x7f8444104690 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f8443fff700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8444104690 0x7f84441a0be0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f843c007ee0 tx=0x7f843c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f84417fa700 1 -- 172.21.15.112:0/494611481 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c00dbd0 con 0x7f8444104690 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84441a1550 con 0x7f8444104690 2024-04-25T10:49:38.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f84417fa700 1 -- 172.21.15.112:0/494611481 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c00dd70 con 0x7f8444104690 2024-04-25T10:49:38.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f84417fa700 1 -- 172.21.15.112:0/494611481 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c005990 con 0x7f8444104690 2024-04-25T10:49:38.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.574+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84441a1a40 con 0x7f8444104690 2024-04-25T10:49:38.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.575+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8444051190 con 0x7f8444104690 2024-04-25T10:49:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.576+0000 7f84417fa700 1 -- 172.21.15.112:0/494611481 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f843c01a020 con 0x7f8444104690 2024-04-25T10:49:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.577+0000 7f84417fa700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84300afb80 0x7f84300b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:49:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.577+0000 7f84417fa700 1 -- 172.21.15.112:0/494611481 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f843c0dd710 con 0x7f8444104690 2024-04-25T10:49:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.577+0000 7f84437fe700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84300afb80 0x7f84300b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:49:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.578+0000 7f84437fe700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84300afb80 0x7f84300b2090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f844406bfd0 tx=0x7f8434009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:49:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.582+0000 7f84417fa700 1 -- 172.21.15.112:0/494611481 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f843c010070 con 0x7f8444104690 2024-04-25T10:49:38.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:38 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi019 2024-04-25T10:49:38.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:38 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-25T10:49:38.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:38.789+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8444063ad0 con 0x7f84300afb80 2024-04-25T10:49:38.791 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:38 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi019 2024-04-25T10:49:38.791 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:38 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-25T10:49:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:38 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi019 2024-04-25T10:49:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:38 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-25T10:49:39.703 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:39 smithi112 ceph-mon[32799]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:39.703 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:39 smithi112 ceph-mon[32799]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T10:49:39.703 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:49:39.704 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:49:39.704 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:39 smithi169 ceph-mon[33049]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:39 smithi169 ceph-mon[33049]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T10:49:39.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:49:39.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:49:39.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:39 smithi019 ceph-mon[29209]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:39 smithi019 ceph-mon[29209]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T10:49:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:49:39.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:49:39.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:40.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:40 smithi169 ceph-mon[33049]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:49:40.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:40 smithi112 ceph-mon[32799]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:49:40.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:40 smithi019 ceph-mon[29209]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:49:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:41 smithi169 ceph-mon[33049]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:41.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:41 smithi112 ceph-mon[32799]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:41 smithi019 ceph-mon[29209]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:42.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:42 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/2810342620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]: dispatch 2024-04-25T10:49:42.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:42 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]: dispatch 2024-04-25T10:49:42.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:42 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]': finished 2024-04-25T10:49:42.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:42 smithi112 ceph-mon[32799]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T10:49:42.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:49:42.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:42 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/3937274513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:49:42.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:42 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/2810342620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]: dispatch 2024-04-25T10:49:42.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:42 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]: dispatch 2024-04-25T10:49:42.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:42 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]': finished 2024-04-25T10:49:42.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:42 smithi019 ceph-mon[29209]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T10:49:42.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:49:42.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:42 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/3937274513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:49:43.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:42 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/2810342620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]: dispatch 2024-04-25T10:49:43.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:42 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]: dispatch 2024-04-25T10:49:43.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:42 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1325203b-15e0-415c-a705-0e6b38ef714a"}]': finished 2024-04-25T10:49:43.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:42 smithi169 ceph-mon[33049]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T10:49:43.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:49:43.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:42 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/3937274513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:49:44.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:43 smithi169 ceph-mon[33049]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:44.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:43 smithi112 ceph-mon[32799]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:44.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:43 smithi019 ceph-mon[29209]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:45.974 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:45 smithi112 ceph-mon[32799]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:46.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:45 smithi169 ceph-mon[33049]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:46.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:45 smithi019 ceph-mon[29209]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:47.988 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:47 smithi112 ceph-mon[32799]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:48.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:47 smithi169 ceph-mon[33049]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:48.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:47 smithi019 ceph-mon[29209]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:50.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:49 smithi169 ceph-mon[33049]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:50.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:49 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T10:49:50.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:49 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:50.028 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:49 smithi112 ceph-mon[32799]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:50.028 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:49 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T10:49:50.028 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:49 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:50.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:49 smithi019 ceph-mon[29209]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:50.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:49 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T10:49:50.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:49 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:50.860 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:50 smithi112 ceph-mon[32799]: Deploying daemon osd.7 on smithi112 2024-04-25T10:49:51.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:50 smithi169 ceph-mon[33049]: Deploying daemon osd.7 on smithi112 2024-04-25T10:49:51.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:50 smithi019 ceph-mon[29209]: Deploying daemon osd.7 on smithi112 2024-04-25T10:49:52.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:51 smithi169 ceph-mon[33049]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:52.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:51 smithi019 ceph-mon[29209]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:52.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:51 smithi112 ceph-mon[32799]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:54.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:53 smithi169 ceph-mon[33049]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:54.008 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:53 smithi112 ceph-mon[32799]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:54.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:53 smithi019 ceph-mon[29209]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:56.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:55 smithi169 ceph-mon[33049]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:56.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:56.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:56.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:55 smithi112 ceph-mon[32799]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:56.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:56.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:56.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:56.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:55 smithi019 ceph-mon[29209]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:56.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:56.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:56.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:56 smithi112 ceph-mon[32799]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:57.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:56 smithi169 ceph-mon[33049]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:57.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:56 smithi019 ceph-mon[29209]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.759+0000 7f84417fa700 1 -- 172.21.15.112:0/494611481 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8444063ad0 con 0x7f84300afb80 2024-04-25T10:49:58.761 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 7 on host 'smithi112' 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84300afb80 msgr2=0x7f84300b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84300afb80 0x7f84300b2090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f844406bfd0 tx=0x7f8434009190 comp rx=0 tx=0).stop 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8444104690 msgr2=0x7f84441a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8444104690 0x7f84441a0be0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f843c007ee0 tx=0x7f843c00d3b0 comp rx=0 tx=0).stop 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 shutdown_connections 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84300afb80 0x7f84300b2090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8444104690 0x7f84441a0be0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84441033b0 0x7f8444199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 --2- 172.21.15.112:0/494611481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8444103d00 0x7f8444199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:49:58.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.761+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 >> 172.21.15.112:0/494611481 conn(0x7f84440fdef0 msgr2=0x7f84441075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:49:58.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.762+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 shutdown_connections 2024-04-25T10:49:58.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:49:58.762+0000 7f844a030700 1 -- 172.21.15.112:0/494611481 wait complete. 2024-04-25T10:49:59.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:58 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:59.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:59.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:59.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:59.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:58 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:59.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:49:59.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:49:59.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:49:59.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:49:59.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:58 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:49:59.374 DEBUG:teuthology.orchestra.run.smithi112:osd.7> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T10:49:59.377 INFO:tasks.cephadm:Deploying osd.8 on smithi169 with /dev/vg_nvme/lv_2... 2024-04-25T10:49:59.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T10:49:59.624 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:50:00.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:59 smithi169 ceph-mon[33049]: from='osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:50:00.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:49:59 smithi169 ceph-mon[33049]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:50:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:59 smithi112 ceph-mon[32799]: from='osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:50:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:49:59 smithi112 ceph-mon[32799]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:50:00.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:59 smithi019 ceph-mon[29209]: from='osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:50:00.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:49:59 smithi019 ceph-mon[29209]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:50:01.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:00 smithi169 ceph-mon[33049]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T10:50:01.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:00 smithi169 ceph-mon[33049]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T10:50:01.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:00 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:01.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:00 smithi169 ceph-mon[33049]: from='osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:50:01.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:00 smithi169 ceph-mon[33049]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:50:01.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:00 smithi169 ceph-mon[33049]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:50:01.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:00 smithi169 ceph-mon[33049]: overall HEALTH_OK 2024-04-25T10:50:01.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-mon[32799]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T10:50:01.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-mon[32799]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T10:50:01.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:01.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-mon[32799]: from='osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:50:01.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-mon[32799]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:50:01.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-mon[32799]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:50:01.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-mon[32799]: overall HEALTH_OK 2024-04-25T10:50:01.060 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T10:50:00.785+0000 7f12905f2700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:50:01.060 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 10:50:00 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T10:50:00.796+0000 7f128c3a7700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:50:01.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:00 smithi019 ceph-mon[29209]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T10:50:01.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:00 smithi019 ceph-mon[29209]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T10:50:01.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:00 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:01.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:00 smithi019 ceph-mon[29209]: from='osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:50:01.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:00 smithi019 ceph-mon[29209]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:50:01.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:00 smithi019 ceph-mon[29209]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:50:01.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:00 smithi019 ceph-mon[29209]: overall HEALTH_OK 2024-04-25T10:50:01.577 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T10:50:01.577 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T10:50:01.577 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T10:50:01.578 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T10:50:01.578 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115672 s, 907 MB/s 2024-04-25T10:50:01.578 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T10:50:01.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:01 smithi112 ceph-mon[32799]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:50:01.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:01 smithi112 ceph-mon[32799]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T10:50:01.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:01 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:01.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:01 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:01.900 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:01 smithi169 ceph-mon[33049]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:50:01.900 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:01 smithi169 ceph-mon[33049]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T10:50:01.900 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:01 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:01.900 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:01 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:02.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:01 smithi019 ceph-mon[29209]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:50:02.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:01 smithi019 ceph-mon[29209]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T10:50:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:01 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:02.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:01 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:02.187 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-04-25T10:50:02.434 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] boot 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi112 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T10:50:03.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:03.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:03.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:02 smithi112 ceph-mon[32799]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] boot 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi112 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:02 smithi019 ceph-mon[29209]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: osd.7 [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] boot 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi112 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:03.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:02 smithi169 ceph-mon[33049]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 -- 172.21.15.169:0/1393919552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104bb0 msgr2=0x7f7f6010b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 --2- 172.21.15.169:0/1393919552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104bb0 0x7f7f6010b460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800b0a0 tx=0x7f7f5800b3b0 comp rx=0 tx=0).stop 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 -- 172.21.15.169:0/1393919552 shutdown_connections 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 --2- 172.21.15.169:0/1393919552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104bb0 0x7f7f6010b460 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 --2- 172.21.15.169:0/1393919552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f601030e0 0x7f7f601034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 --2- 172.21.15.169:0/1393919552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f60104210 0x7f7f60104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 -- 172.21.15.169:0/1393919552 >> 172.21.15.169:0/1393919552 conn(0x7f7f600fe740 msgr2=0x7f7f60100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 -- 172.21.15.169:0/1393919552 shutdown_connections 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.965+0000 7f7f66960700 1 -- 172.21.15.169:0/1393919552 wait complete. 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.966+0000 7f7f66960700 1 Processor -- start 2024-04-25T10:50:03.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.966+0000 7f7f66960700 1 -- start start 2024-04-25T10:50:03.969 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.966+0000 7f7f66960700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f601030e0 0x7f7f60199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f66960700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104210 0x7f7f60199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f66960700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f60104bb0 0x7f7f601a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f66960700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6010ebd0 con 0x7f7f601030e0 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f66960700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6010ea50 con 0x7f7f60104bb0 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f66960700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6010ed50 con 0x7f7f60104210 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104210 0x7f7f60199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104210 0x7f7f60199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41624/0 (socket says 172.21.15.169:41624) 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 -- 172.21.15.169:0/4061445270 learned_addr learned my addr 172.21.15.169:0/4061445270 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f64efd700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f60104bb0 0x7f7f601a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5ffff700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f601030e0 0x7f7f60199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:03.970 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 -- 172.21.15.169:0/4061445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f60104bb0 msgr2=0x7f7f601a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f60104bb0 0x7f7f601a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 -- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f601030e0 msgr2=0x7f7f60199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f601030e0 0x7f7f60199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 -- 172.21.15.169:0/4061445270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f601a11e0 con 0x7f7f60104210 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5ffff700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f601030e0 0x7f7f60199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5f7fe700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104210 0x7f7f60199bc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b530 tx=0x7f7f5400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5d7fa700 1 -- 172.21.15.169:0/4061445270 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54015020 con 0x7f7f60104210 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f601a14d0 con 0x7f7f60104210 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5d7fa700 1 -- 172.21.15.169:0/4061445270 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f54011ce0 con 0x7f7f60104210 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f5d7fa700 1 -- 172.21.15.169:0/4061445270 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54010e40 con 0x7f7f60104210 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.967+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f601a1ae0 con 0x7f7f60104210 2024-04-25T10:50:03.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.968+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f60051190 con 0x7f7f60104210 2024-04-25T10:50:03.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.970+0000 7f7f5d7fa700 1 -- 172.21.15.169:0/4061445270 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7f54018020 con 0x7f7f60104210 2024-04-25T10:50:03.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.970+0000 7f7f5d7fa700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7f480afb80 0x7f7f480b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:03.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.971+0000 7f7f5d7fa700 1 -- 172.21.15.169:0/4061445270 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f7f540ddeb0 con 0x7f7f60104210 2024-04-25T10:50:03.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.971+0000 7f7f5ffff700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7f480afb80 0x7f7f480b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:03.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.971+0000 7f7f5ffff700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7f480afb80 0x7f7f480b2090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005fd0 tx=0x7f7f50005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:50:03.975 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:03.974+0000 7f7f5d7fa700 1 -- 172.21.15.169:0/4061445270 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f5401f020 con 0x7f7f60104210 2024-04-25T10:50:04.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:04.179+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7f60075980 con 0x7f7f480afb80 2024-04-25T10:50:04.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:04 smithi169 ceph-mon[33049]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T10:50:04.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:04 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:04.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:04 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:04.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:04 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:04.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:04 smithi112 ceph-mon[32799]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T10:50:04.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:04 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:04.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:04 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:04.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:04 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:04.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:04 smithi019 ceph-mon[29209]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T10:50:04.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:04 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:04.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:04 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:04.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:04 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:05 smithi169 ceph-mon[33049]: pgmap v177: 1 pgs: 1 peering; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:05 smithi169 ceph-mon[33049]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:05.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:05 smithi112 ceph-mon[32799]: pgmap v177: 1 pgs: 1 peering; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:05.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:05 smithi112 ceph-mon[32799]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:05.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:05 smithi019 ceph-mon[29209]: pgmap v177: 1 pgs: 1 peering; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:05.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:05 smithi019 ceph-mon[29209]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:07 smithi169 ceph-mon[33049]: pgmap v178: 1 pgs: 1 peering; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:07 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/677995759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]: dispatch 2024-04-25T10:50:07.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:07 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]: dispatch 2024-04-25T10:50:07.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:07 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]': finished 2024-04-25T10:50:07.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:07 smithi169 ceph-mon[33049]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T10:50:07.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:07.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:07 smithi112 ceph-mon[32799]: pgmap v178: 1 pgs: 1 peering; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:07.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:07 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/677995759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]: dispatch 2024-04-25T10:50:07.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:07 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]: dispatch 2024-04-25T10:50:07.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:07 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]': finished 2024-04-25T10:50:07.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:07 smithi112 ceph-mon[32799]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T10:50:07.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:07.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:07 smithi019 ceph-mon[29209]: pgmap v178: 1 pgs: 1 peering; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:07.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:07 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/677995759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]: dispatch 2024-04-25T10:50:07.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:07 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]: dispatch 2024-04-25T10:50:07.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:07 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45140686-3fd7-44dd-a246-d67730c2b8c3"}]': finished 2024-04-25T10:50:07.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:07 smithi019 ceph-mon[29209]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T10:50:07.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:08.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:08 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/1211630274' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:50:08.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:08 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/1211630274' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:50:08.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:08 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/1211630274' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:50:09.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:09 smithi169 ceph-mon[33049]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:09.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:09 smithi112 ceph-mon[32799]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:09.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:09 smithi019 ceph-mon[29209]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:11.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:11 smithi169 ceph-mon[33049]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:11 smithi112 ceph-mon[32799]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:11.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:11 smithi019 ceph-mon[29209]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:13.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:13 smithi169 ceph-mon[33049]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:13.791 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:13 smithi112 ceph-mon[32799]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:13.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:13 smithi019 ceph-mon[29209]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:15.736 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:15 smithi169 ceph-mon[33049]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:15.737 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:15 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T10:50:15.737 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:15 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:15.737 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:15 smithi169 ceph-mon[33049]: Deploying daemon osd.8 on smithi169 2024-04-25T10:50:15.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:15 smithi112 ceph-mon[32799]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:15.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:15 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T10:50:15.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:15 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:15.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:15 smithi112 ceph-mon[32799]: Deploying daemon osd.8 on smithi169 2024-04-25T10:50:15.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:15 smithi019 ceph-mon[29209]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:15.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:15 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T10:50:15.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:15 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:15.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:15 smithi019 ceph-mon[29209]: Deploying daemon osd.8 on smithi169 2024-04-25T10:50:17.680 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:17 smithi169 ceph-mon[33049]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:17.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:17 smithi112 ceph-mon[32799]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:17.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:17 smithi019 ceph-mon[29209]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:19.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:19 smithi169 ceph-mon[33049]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:19.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:19 smithi112 ceph-mon[32799]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:19 smithi019 ceph-mon[29209]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:21 smithi169 ceph-mon[33049]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:21 smithi112 ceph-mon[32799]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:21.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:21 smithi019 ceph-mon[29209]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:21.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:21.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:21.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:23 smithi169 ceph-mon[33049]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:23.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:23.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:23.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:23 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:23 smithi112 ceph-mon[32799]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:23 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:23 smithi019 ceph-mon[29209]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:23.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:23.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:23.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:23.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:23 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:24.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:23.999+0000 7f7f5d7fa700 1 -- 172.21.15.169:0/4061445270 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7f60075980 con 0x7f7f480afb80 2024-04-25T10:50:24.000 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 8 on host 'smithi169' 2024-04-25T10:50:24.002 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7f480afb80 msgr2=0x7f7f480b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7f480afb80 0x7f7f480b2090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005fd0 tx=0x7f7f50005f00 comp rx=0 tx=0).stop 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104210 msgr2=0x7f7f60199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104210 0x7f7f60199bc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b530 tx=0x7f7f5400b840 comp rx=0 tx=0).stop 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 shutdown_connections 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7f480afb80 0x7f7f480b2090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f60104bb0 0x7f7f601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f601030e0 0x7f7f60199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 --2- 172.21.15.169:0/4061445270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60104210 0x7f7f60199bc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 >> 172.21.15.169:0/4061445270 conn(0x7f7f600fe740 msgr2=0x7f7f60107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 shutdown_connections 2024-04-25T10:50:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:50:24.002+0000 7f7f66960700 1 -- 172.21.15.169:0/4061445270 wait complete. 2024-04-25T10:50:24.563 DEBUG:teuthology.orchestra.run.smithi169:osd.8> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T10:50:24.566 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-04-25T10:50:24.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T10:50:24.825 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:50:25.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:25.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:25.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:24 smithi169 ceph-mon[33049]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:25.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:25.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:24 smithi169 ceph-mon[33049]: from='osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:50:25.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:24 smithi169 ceph-mon[33049]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:50:25.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:25.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:25.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:24 smithi019 ceph-mon[29209]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:25.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:25.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:24 smithi019 ceph-mon[29209]: from='osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:50:25.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:24 smithi019 ceph-mon[29209]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:50:25.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:25.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:25.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:24 smithi112 ceph-mon[32799]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:25.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:25.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:24 smithi112 ceph-mon[32799]: from='osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:50:25.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:24 smithi112 ceph-mon[32799]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:50:26.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:25 smithi169 ceph-mon[33049]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T10:50:26.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:25 smithi169 ceph-mon[33049]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T10:50:26.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:25 smithi169 ceph-mon[33049]: from='osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:50:26.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:25 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:26.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:25 smithi169 ceph-mon[33049]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:50:26.252 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 10:50:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T10:50:25.967+0000 7f2b978f4700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T10:50:26.252 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 10:50:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T10:50:25.981+0000 7f2b936a9700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:50:26.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:25 smithi019 ceph-mon[29209]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T10:50:26.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:25 smithi019 ceph-mon[29209]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T10:50:26.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:25 smithi019 ceph-mon[29209]: from='osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:50:26.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:25 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:26.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:25 smithi019 ceph-mon[29209]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:50:26.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:25 smithi112 ceph-mon[32799]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T10:50:26.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:25 smithi112 ceph-mon[32799]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T10:50:26.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:25 smithi112 ceph-mon[32799]: from='osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:50:26.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:25 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:26.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:25 smithi112 ceph-mon[32799]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:50:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T10:50:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T10:50:27.012 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T10:50:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T10:50:27.012 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111947 s, 937 MB/s 2024-04-25T10:50:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T10:50:27.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:26 smithi169 ceph-mon[33049]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:50:27.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:26 smithi169 ceph-mon[33049]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:27.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:26 smithi169 ceph-mon[33049]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T10:50:27.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:26 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:27.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:26 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:27.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:26 smithi112 ceph-mon[32799]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:50:27.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:26 smithi112 ceph-mon[32799]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:27.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:26 smithi112 ceph-mon[32799]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T10:50:27.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:26 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:27.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:26 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:27.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:26 smithi019 ceph-mon[29209]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:50:27.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:26 smithi019 ceph-mon[29209]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:27.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:26 smithi019 ceph-mon[29209]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T10:50:27.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:26 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:27.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:26 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:27.704 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-04-25T10:50:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:50:28.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] boot 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi169 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T10:50:28.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:28.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:28.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:50:28.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] boot 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi169 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:28.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:28.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:50:28.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:50:28.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:28.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: osd.8 [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] boot 2024-04-25T10:50:28.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T10:50:28.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:50:28.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T10:50:28.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi169 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:28.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:28.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:28.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.642+0000 7f123deb3700 1 -- 172.21.15.19:0/2178478855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104bb0 msgr2=0x7f123810b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.642+0000 7f123deb3700 1 --2- 172.21.15.19:0/2178478855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104bb0 0x7f123810b460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f123000b0a0 tx=0x7f123000b3b0 comp rx=0 tx=0).stop 2024-04-25T10:50:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.642+0000 7f123deb3700 1 -- 172.21.15.19:0/2178478855 shutdown_connections 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.642+0000 7f123deb3700 1 --2- 172.21.15.19:0/2178478855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104bb0 0x7f123810b460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.642+0000 7f123deb3700 1 --2- 172.21.15.19:0/2178478855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381030e0 0x7f12381034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.642+0000 7f123deb3700 1 --2- 172.21.15.19:0/2178478855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1238104210 0x7f1238104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.642+0000 7f123deb3700 1 -- 172.21.15.19:0/2178478855 >> 172.21.15.19:0/2178478855 conn(0x7f12380fe740 msgr2=0x7f1238100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.643+0000 7f123deb3700 1 -- 172.21.15.19:0/2178478855 shutdown_connections 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.643+0000 7f123deb3700 1 -- 172.21.15.19:0/2178478855 wait complete. 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.643+0000 7f123deb3700 1 Processor -- start 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.643+0000 7f123deb3700 1 -- start start 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f123deb3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381030e0 0x7f123810da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f123deb3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104210 0x7f123810c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f123deb3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238104bb0 0x7f123810c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f123deb3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f123810fcc0 con 0x7f1238104210 2024-04-25T10:50:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f123deb3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f123810fb40 con 0x7f1238104bb0 2024-04-25T10:50:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f123deb3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f123810fe40 con 0x7f12381030e0 2024-04-25T10:50:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f12377fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381030e0 0x7f123810da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104210 0x7f123810c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f12377fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381030e0 0x7f123810da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:56190/0 (socket says 172.21.15.19:56190) 2024-04-25T10:50:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104210 0x7f123810c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33082/0 (socket says 172.21.15.19:33082) 2024-04-25T10:50:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f12377fe700 1 -- 172.21.15.19:0/3947300037 learned_addr learned my addr 172.21.15.19:0/3947300037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:50:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:29 smithi019 ceph-mon[29209]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:29 smithi019 ceph-mon[29209]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T10:50:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1237fff700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238104bb0 0x7f123810c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 -- 172.21.15.19:0/3947300037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381030e0 msgr2=0x7f123810da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381030e0 0x7f123810da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 -- 172.21.15.19:0/3947300037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238104bb0 msgr2=0x7f123810c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238104bb0 0x7f123810c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 -- 172.21.15.19:0/3947300037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123810cce0 con 0x7f1238104210 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.644+0000 7f1236ffd700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104210 0x7f123810c0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f122c00d950 tx=0x7f122c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.645+0000 7f1234ff9700 1 -- 172.21.15.19:0/3947300037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c016070 con 0x7f1238104210 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.645+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12381a4810 con 0x7f1238104210 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.645+0000 7f1234ff9700 1 -- 172.21.15.19:0/3947300037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c00fd40 con 0x7f1238104210 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.645+0000 7f1234ff9700 1 -- 172.21.15.19:0/3947300037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c00e3b0 con 0x7f1238104210 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.645+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381a4c70 con 0x7f1238104210 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.646+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1238051190 con 0x7f1238104210 2024-04-25T10:50:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.649+0000 7f1234ff9700 1 -- 172.21.15.19:0/3947300037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f122c019070 con 0x7f1238104210 2024-04-25T10:50:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.649+0000 7f1234ff9700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12200afb80 0x7f12200b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.649+0000 7f12377fe700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12200afb80 0x7f12200b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.649+0000 7f1234ff9700 1 -- 172.21.15.19:0/3947300037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f122c0de170 con 0x7f1238104210 2024-04-25T10:50:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.650+0000 7f12377fe700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12200afb80 0x7f12200b2090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f122800acb0 tx=0x7f122800a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:50:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.653+0000 7f1234ff9700 1 -- 172.21.15.19:0/3947300037 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f122c0a7700 con 0x7f1238104210 2024-04-25T10:50:29.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:29 smithi169 ceph-mon[33049]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:29.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:29 smithi169 ceph-mon[33049]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T10:50:29.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:29 smithi112 ceph-mon[32799]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:50:29.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:29 smithi112 ceph-mon[32799]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T10:50:29.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:29.871+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1238063ad0 con 0x7f12200afb80 2024-04-25T10:50:30.728 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:30.728 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:30.728 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:30.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:30.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:30.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:30.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:30.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:30.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:31.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:31 smithi169 ceph-mon[33049]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:31.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:31 smithi169 ceph-mon[33049]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:31.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:31 smithi019 ceph-mon[29209]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:31.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:31 smithi019 ceph-mon[29209]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:31.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:31 smithi112 ceph-mon[32799]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:31.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:31 smithi112 ceph-mon[32799]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:33.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:33 smithi019 ceph-mon[29209]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3168090382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06"}]: dispatch 2024-04-25T10:50:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3168090382' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06"}]': finished 2024-04-25T10:50:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:33 smithi019 ceph-mon[29209]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T10:50:33.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:33.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:33 smithi169 ceph-mon[33049]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:33.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3168090382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06"}]: dispatch 2024-04-25T10:50:33.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3168090382' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06"}]': finished 2024-04-25T10:50:33.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:33 smithi169 ceph-mon[33049]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T10:50:33.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:33 smithi112 ceph-mon[32799]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3168090382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06"}]: dispatch 2024-04-25T10:50:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3168090382' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06"}]': finished 2024-04-25T10:50:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:33 smithi112 ceph-mon[32799]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T10:50:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:34.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:34 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/628225749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:50:34.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:34 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/628225749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:50:34.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:34 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/628225749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:50:35.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:35 smithi169 ceph-mon[33049]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:50:35.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:35 smithi112 ceph-mon[32799]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:50:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:35 smithi019 ceph-mon[29209]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:50:37.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:37 smithi019 ceph-mon[29209]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:50:37.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:37 smithi169 ceph-mon[33049]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:50:37.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:37 smithi112 ceph-mon[32799]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:50:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:39 smithi169 ceph-mon[33049]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:50:39.756 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:39 smithi019 ceph-mon[29209]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:50:39.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:39 smithi112 ceph-mon[32799]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:50:41.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:41 smithi019 ceph-mon[29209]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:50:41.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T10:50:41.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:41 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:41 smithi169 ceph-mon[33049]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:50:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T10:50:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:41 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:41.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:41 smithi112 ceph-mon[32799]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:50:41.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T10:50:41.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:41 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:42.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:42 smithi169 ceph-mon[33049]: Deploying daemon osd.9 on smithi019 2024-04-25T10:50:42.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:42 smithi112 ceph-mon[32799]: Deploying daemon osd.9 on smithi019 2024-04-25T10:50:42.831 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:42 smithi019 ceph-mon[29209]: Deploying daemon osd.9 on smithi019 2024-04-25T10:50:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:43 smithi169 ceph-mon[33049]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:50:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:43 smithi112 ceph-mon[32799]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:50:43.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:43 smithi019 ceph-mon[29209]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:50:45.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:45 smithi019 ceph-mon[29209]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:45.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:45 smithi169 ceph-mon[33049]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:45 smithi112 ceph-mon[32799]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:47.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:47 smithi169 ceph-mon[33049]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:47 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:47 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:47 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:47 smithi112 ceph-mon[32799]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:47 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:47 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:47 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:47.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:47 smithi019 ceph-mon[29209]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:47.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:47 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:47.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:47 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:47.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:47 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:49.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:49 smithi019 ceph-mon[29209]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:49.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:49 smithi169 ceph-mon[33049]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:49.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:49 smithi112 ceph-mon[32799]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.683+0000 7f1234ff9700 1 -- 172.21.15.19:0/3947300037 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1238063ad0 con 0x7f12200afb80 2024-04-25T10:50:50.685 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12200afb80 msgr2=0x7f12200b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12200afb80 0x7f12200b2090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f122800acb0 tx=0x7f122800a400 comp rx=0 tx=0).stop 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104210 msgr2=0x7f123810c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104210 0x7f123810c0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f122c00d950 tx=0x7f122c00dc60 comp rx=0 tx=0).stop 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 shutdown_connections 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12200afb80 0x7f12200b2090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238104bb0 0x7f123810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381030e0 0x7f123810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 --2- 172.21.15.19:0/3947300037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1238104210 0x7f123810c0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 >> 172.21.15.19:0/3947300037 conn(0x7f12380fe740 msgr2=0x7f1238107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:50:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 shutdown_connections 2024-04-25T10:50:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:50:50.685+0000 7f123deb3700 1 -- 172.21.15.19:0/3947300037 wait complete. 2024-04-25T10:50:51.312 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T10:50:51.314 INFO:tasks.cephadm:Deploying osd.10 on smithi112 with /dev/vg_nvme/lv_1... 2024-04-25T10:50:51.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T10:50:51.561 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:50:51.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:51 smithi112 ceph-mon[32799]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:51.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:51.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:51.914 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:51 smithi112 ceph-mon[32799]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T10:50:51.915 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:51.915 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:51 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:51 smithi169 ceph-mon[33049]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:52.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:51 smithi169 ceph-mon[33049]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T10:50:52.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:51 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:51 smithi019 ceph-mon[29209]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:52.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:51 smithi019 ceph-mon[29209]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T10:50:52.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:51 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:52.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:52 smithi019 ceph-mon[29209]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T10:50:52.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:52 smithi019 ceph-mon[29209]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T10:50:52.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:52 smithi019 ceph-mon[29209]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:50:52.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:52 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:52.987 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:52 smithi112 ceph-mon[32799]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T10:50:52.987 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:52 smithi112 ceph-mon[32799]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T10:50:52.987 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:52 smithi112 ceph-mon[32799]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:50:52.987 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:52 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:53.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:52 smithi169 ceph-mon[33049]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T10:50:53.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:52 smithi169 ceph-mon[33049]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T10:50:53.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:52 smithi169 ceph-mon[33049]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T10:50:53.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:52 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:53.697 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T10:50:53.697 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T10:50:53.697 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T10:50:53.697 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T10:50:53.697 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117628 s, 891 MB/s 2024-04-25T10:50:53.697 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T10:50:53.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-mon[29209]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:53.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-mon[29209]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:50:53.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-mon[29209]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T10:50:53.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:53.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:53.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-mon[29209]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' 2024-04-25T10:50:53.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:53.862 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T10:50:53.601+0000 7fd902ff7700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T10:50:53.862 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 10:50:53 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T10:50:53.703+0000 7fd8fada4700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:50:54.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:53 smithi169 ceph-mon[33049]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:54.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:53 smithi169 ceph-mon[33049]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:50:54.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:53 smithi169 ceph-mon[33049]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T10:50:54.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:54.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:54.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:53 smithi169 ceph-mon[33049]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' 2024-04-25T10:50:54.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:54.014 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:53 smithi112 ceph-mon[32799]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:54.014 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:53 smithi112 ceph-mon[32799]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T10:50:54.014 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:53 smithi112 ceph-mon[32799]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T10:50:54.014 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:54.014 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:54.014 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:53 smithi112 ceph-mon[32799]: from='osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159]' entity='osd.9' 2024-04-25T10:50:54.014 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:54.333 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-04-25T10:50:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:50:54.951 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:54 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:50:54.951 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:54 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:50:54.952 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:55.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:54 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:50:55.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:54 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:50:55.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:55.019 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:54 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:50:55.019 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:54 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:50:55.019 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:55.941 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] boot 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:55.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:55.943 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:55.943 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:55.943 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:55.943 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:55 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:56.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] boot 2024-04-25T10:50:56.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:56.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:56.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:55 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:50:56.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: osd.9 [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] boot 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:50:56.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:56.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:50:56.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:55 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:50:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 -- 172.21.15.112:0/843543748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a681048b0 msgr2=0x7f8a6810c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 --2- 172.21.15.112:0/843543748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a681048b0 0x7f8a6810c5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000b0a0 tx=0x7f8a6000b3b0 comp rx=0 tx=0).stop 2024-04-25T10:50:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 -- 172.21.15.112:0/843543748 shutdown_connections 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 --2- 172.21.15.112:0/843543748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a681048b0 0x7f8a6810c5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 --2- 172.21.15.112:0/843543748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a68105880 0x7f8a68105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 --2- 172.21.15.112:0/843543748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68103e30 0x7f8a68104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 -- 172.21.15.112:0/843543748 >> 172.21.15.112:0/843543748 conn(0x7f8a680ff8a0 msgr2=0x7f8a68101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 -- 172.21.15.112:0/843543748 shutdown_connections 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 -- 172.21.15.112:0/843543748 wait complete. 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.305+0000 7f8a704f8700 1 Processor -- start 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a704f8700 1 -- start start 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a704f8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a68103e30 0x7f8a6819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a704f8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a681048b0 0x7f8a6819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a704f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68105880 0x7f8a681a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a704f8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6810fe30 con 0x7f8a68105880 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a704f8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6810fcb0 con 0x7f8a68103e30 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a704f8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6810ffb0 con 0x7f8a681048b0 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a68103e30 0x7f8a6819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a68103e30 0x7f8a6819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58434/0 (socket says 172.21.15.112:58434) 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 -- 172.21.15.112:0/1535707026 learned_addr learned my addr 172.21.15.112:0/1535707026 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6ea95700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68105880 0x7f8a681a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6da93700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a681048b0 0x7f8a6819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 -- 172.21.15.112:0/1535707026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a681048b0 msgr2=0x7f8a6819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a681048b0 0x7f8a6819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 -- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68105880 msgr2=0x7f8a681a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:50:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68105880 0x7f8a681a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:50:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 -- 172.21.15.112:0/1535707026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a681a2440 con 0x7f8a68103e30 2024-04-25T10:50:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a6e294700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a68103e30 0x7f8a6819a8e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400d950 tx=0x7f8a6400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:50:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.306+0000 7f8a5f7fe700 1 -- 172.21.15.112:0/1535707026 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64017070 con 0x7f8a68103e30 2024-04-25T10:50:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.307+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a681a2730 con 0x7f8a68103e30 2024-04-25T10:50:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.307+0000 7f8a5f7fe700 1 -- 172.21.15.112:0/1535707026 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a64009730 con 0x7f8a68103e30 2024-04-25T10:50:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.307+0000 7f8a5f7fe700 1 -- 172.21.15.112:0/1535707026 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64003a50 con 0x7f8a68103e30 2024-04-25T10:50:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.307+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a681a2cf0 con 0x7f8a68103e30 2024-04-25T10:50:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.308+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a68051190 con 0x7f8a68103e30 2024-04-25T10:50:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.310+0000 7f8a5f7fe700 1 -- 172.21.15.112:0/1535707026 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8a64003bf0 con 0x7f8a68103e30 2024-04-25T10:50:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.310+0000 7f8a5f7fe700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a540afb80 0x7f8a540b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:50:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.311+0000 7f8a5f7fe700 1 -- 172.21.15.112:0/1535707026 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f8a640de6f0 con 0x7f8a68103e30 2024-04-25T10:50:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.311+0000 7f8a6da93700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a540afb80 0x7f8a540b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:50:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.311+0000 7f8a6da93700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a540afb80 0x7f8a540b2090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:50:56.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.315+0000 7f8a5f7fe700 1 -- 172.21.15.112:0/1535707026 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a64020030 con 0x7f8a68103e30 2024-04-25T10:50:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:50:56.521+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8a6802e8f0 con 0x7f8a540afb80 2024-04-25T10:50:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:56 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi019 2024-04-25T10:50:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:56 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T10:50:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:56 smithi112 ceph-mon[32799]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T10:50:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:56.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:56 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:57.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:56 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi019 2024-04-25T10:50:57.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:56 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T10:50:57.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:56 smithi169 ceph-mon[33049]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T10:50:57.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:57.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:57.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:56 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:57.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:56 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi019 2024-04-25T10:50:57.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:56 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T10:50:57.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:56 smithi019 ceph-mon[29209]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T10:50:57.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:50:57.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:50:57.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:56 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:50:58.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:57 smithi169 ceph-mon[33049]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:50:58.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:57 smithi169 ceph-mon[33049]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:58.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:57 smithi112 ceph-mon[32799]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:50:58.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:57 smithi112 ceph-mon[32799]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:50:58.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:57 smithi019 ceph-mon[29209]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:50:58.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:57 smithi019 ceph-mon[29209]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:51:00.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:59 smithi169 ceph-mon[33049]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:00.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:59 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/1098996857' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]: dispatch 2024-04-25T10:51:00.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:59 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]: dispatch 2024-04-25T10:51:00.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:59 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]': finished 2024-04-25T10:51:00.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:59 smithi169 ceph-mon[33049]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T10:51:00.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:50:59 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:59 smithi112 ceph-mon[32799]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:59 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/1098996857' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]: dispatch 2024-04-25T10:51:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:59 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]: dispatch 2024-04-25T10:51:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:59 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]': finished 2024-04-25T10:51:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:59 smithi112 ceph-mon[32799]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T10:51:00.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:50:59 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:00.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:59 smithi019 ceph-mon[29209]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:00.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:59 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/1098996857' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]: dispatch 2024-04-25T10:51:00.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:59 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]: dispatch 2024-04-25T10:51:00.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:59 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6301d36c-b3e4-4840-bde6-833e2da34788"}]': finished 2024-04-25T10:51:00.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:59 smithi019 ceph-mon[29209]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T10:51:00.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:50:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:01.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:00 smithi169 ceph-mon[33049]: from='client.? 172.21.15.112:0/2676846612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:51:01.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:00 smithi112 ceph-mon[32799]: from='client.? 172.21.15.112:0/2676846612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:51:01.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:00 smithi019 ceph-mon[29209]: from='client.? 172.21.15.112:0/2676846612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:51:02.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:01 smithi169 ceph-mon[33049]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:02.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:01 smithi112 ceph-mon[32799]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:02.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:01 smithi019 ceph-mon[29209]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:03.974 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:03 smithi112 ceph-mon[32799]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:04.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:03 smithi169 ceph-mon[33049]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:04.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:03 smithi019 ceph-mon[29209]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:05.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:05 smithi112 ceph-mon[32799]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:06.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:05 smithi169 ceph-mon[33049]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:06.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:05 smithi019 ceph-mon[29209]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:07.782 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:07 smithi112 ceph-mon[32799]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:07.782 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T10:51:07.782 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:07 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:08.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:07 smithi169 ceph-mon[33049]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:08.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T10:51:08.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:07 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:07 smithi019 ceph-mon[29209]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T10:51:08.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:07 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:08.769 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:08 smithi112 ceph-mon[32799]: Deploying daemon osd.10 on smithi112 2024-04-25T10:51:09.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:08 smithi169 ceph-mon[33049]: Deploying daemon osd.10 on smithi112 2024-04-25T10:51:09.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:08 smithi019 ceph-mon[29209]: Deploying daemon osd.10 on smithi112 2024-04-25T10:51:09.982 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:09 smithi112 ceph-mon[32799]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:10.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:09 smithi169 ceph-mon[33049]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:10.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:09 smithi019 ceph-mon[29209]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:12.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:11 smithi169 ceph-mon[33049]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:12.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:11 smithi112 ceph-mon[32799]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:12.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:11 smithi019 ceph-mon[29209]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:14.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:13 smithi169 ceph-mon[33049]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:14.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:14.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:14.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:14.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:13 smithi112 ceph-mon[32799]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:14.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:14.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:14.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:14.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:13 smithi019 ceph-mon[29209]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:14.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:14.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:14.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:15.955 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:15 smithi112 ceph-mon[32799]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:16.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:15 smithi169 ceph-mon[33049]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:16.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:15 smithi019 ceph-mon[29209]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:16.293 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 10 on host 'smithi112' 2024-04-25T10:51:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.291+0000 7f8a5f7fe700 1 -- 172.21.15.112:0/1535707026 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8a6802e8f0 con 0x7f8a540afb80 2024-04-25T10:51:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.295+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a540afb80 msgr2=0x7f8a540b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.295+0000 7f8a704f8700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a540afb80 0x7f8a540b2090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2024-04-25T10:51:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a68103e30 msgr2=0x7f8a6819a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a68103e30 0x7f8a6819a8e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400d950 tx=0x7f8a6400dc60 comp rx=0 tx=0).stop 2024-04-25T10:51:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 shutdown_connections 2024-04-25T10:51:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a540afb80 0x7f8a540b2090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a68105880 0x7f8a681a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a68103e30 0x7f8a6819a8e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 --2- 172.21.15.112:0/1535707026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a681048b0 0x7f8a6819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 >> 172.21.15.112:0/1535707026 conn(0x7f8a680ff8a0 msgr2=0x7f8a681085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 shutdown_connections 2024-04-25T10:51:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T10:51:16.296+0000 7f8a704f8700 1 -- 172.21.15.112:0/1535707026 wait complete. 2024-04-25T10:51:16.969 DEBUG:teuthology.orchestra.run.smithi112:osd.10> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T10:51:16.971 INFO:tasks.cephadm:Deploying osd.11 on smithi169 with /dev/vg_nvme/lv_1... 2024-04-25T10:51:16.971 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T10:51:17.219 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:51:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:17 smithi169 ceph-mon[33049]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:17 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:17 smithi169 ceph-mon[33049]: from='osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:51:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:17 smithi169 ceph-mon[33049]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:51:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:17 smithi112 ceph-mon[32799]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:17 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:17 smithi112 ceph-mon[32799]: from='osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:51:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:17 smithi112 ceph-mon[32799]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:51:17.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:17 smithi019 ceph-mon[29209]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:17.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:17 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:17.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:17 smithi019 ceph-mon[29209]: from='osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:51:17.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:17 smithi019 ceph-mon[29209]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:51:18.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:18 smithi169 ceph-mon[33049]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T10:51:18.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:18 smithi169 ceph-mon[33049]: from='osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:51:18.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:18 smithi169 ceph-mon[33049]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T10:51:18.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:18 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:18.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:18 smithi169 ceph-mon[33049]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:51:18.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:18 smithi112 ceph-mon[32799]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T10:51:18.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:18 smithi112 ceph-mon[32799]: from='osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:51:18.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:18 smithi112 ceph-mon[32799]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T10:51:18.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:18 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:18.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:18 smithi112 ceph-mon[32799]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:51:18.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:18 smithi019 ceph-mon[29209]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T10:51:18.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:18 smithi019 ceph-mon[29209]: from='osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:51:18.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:18 smithi019 ceph-mon[29209]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T10:51:18.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:18 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:18.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:18 smithi019 ceph-mon[29209]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T10:51:19.310 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T10:51:19.310 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T10:51:19.311 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T10:51:19.311 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T10:51:19.311 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011579 s, 906 MB/s 2024-04-25T10:51:19.311 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T10:51:19.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:19.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:51:19.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T10:51:19.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: from='osd.10 ' entity='osd.10' 2024-04-25T10:51:19.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T10:51:19.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.560 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T10:51:19.354+0000 7f0ac66b8700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T10:51:19.560 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 10:51:19 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T10:51:19.512+0000 7f0abe465700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: from='osd.10 ' entity='osd.10' 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T10:51:19.628 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T10:51:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T10:51:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: from='osd.10 ' entity='osd.10' 2024-04-25T10:51:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T10:51:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:19.963 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-04-25T10:51:20.204 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:51:20.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:51:20.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:51:20.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:20.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] boot 2024-04-25T10:51:20.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T10:51:20.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:20 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:51:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:51:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] boot 2024-04-25T10:51:20.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T10:51:20.811 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:20 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:20.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:51:20.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:51:20.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:20.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: osd.10 [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] boot 2024-04-25T10:51:20.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T10:51:20.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:20 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:51:21.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:21 smithi169 ceph-mon[33049]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:21.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:21 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi112 2024-04-25T10:51:21.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:21 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T10:51:21.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:21 smithi169 ceph-mon[33049]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T10:51:21.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:21 smithi019 ceph-mon[29209]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:21.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:21 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi112 2024-04-25T10:51:21.763 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:21 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T10:51:21.763 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:21 smithi019 ceph-mon[29209]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T10:51:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:21 smithi112 ceph-mon[32799]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:51:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:21 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi112 2024-04-25T10:51:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:21 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T10:51:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:21 smithi112 ceph-mon[32799]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 -- 172.21.15.169:0/2231135236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105340 msgr2=0x7f14a0105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/2231135236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105340 0x7f14a0105750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1488009990 tx=0x7f1488009ca0 comp rx=0 tx=0).stop 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 -- 172.21.15.169:0/2231135236 shutdown_connections 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/2231135236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14a0105ce0 0x7f14a010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/2231135236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a0104210 0x7f14a0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/2231135236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105340 0x7f14a0105750 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 -- 172.21.15.169:0/2231135236 >> 172.21.15.169:0/2231135236 conn(0x7f14a00ff870 msgr2=0x7f14a0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 -- 172.21.15.169:0/2231135236 shutdown_connections 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.932+0000 7f14a5ce6700 1 -- 172.21.15.169:0/2231135236 wait complete. 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 Processor -- start 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 -- start start 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14a0104210 0x7f14a019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:21.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a0105340 0x7f14a019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105ce0 0x7f14a01a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14a010fd70 con 0x7f14a0105340 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14a010fbf0 con 0x7f14a0104210 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f14a5ce6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14a010fef0 con 0x7f14a0105ce0 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f149ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105ce0 0x7f14a01a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f149ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105ce0 0x7f14a01a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50826/0 (socket says 172.21.15.169:50826) 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f149ffff700 1 -- 172.21.15.169:0/1174271885 learned_addr learned my addr 172.21.15.169:0/1174271885 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f149f7fe700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14a0104210 0x7f14a019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.933+0000 7f1497fff700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a0105340 0x7f14a019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149ffff700 1 -- 172.21.15.169:0/1174271885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14a0104210 msgr2=0x7f14a019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149ffff700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14a0104210 0x7f14a019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149ffff700 1 -- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a0105340 msgr2=0x7f14a019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:21.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149ffff700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a0105340 0x7f14a019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149ffff700 1 -- 172.21.15.169:0/1174271885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a01a2420 con 0x7f14a0105ce0 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f1497fff700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a0105340 0x7f14a019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149ffff700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105ce0 0x7f14a01a1da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f149800b970 tx=0x7f149800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149d7fa700 1 -- 172.21.15.169:0/1174271885 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498019070 con 0x7f14a0105ce0 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149d7fa700 1 -- 172.21.15.169:0/1174271885 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f149800cde0 con 0x7f14a0105ce0 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a01a2710 con 0x7f14a0105ce0 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f149d7fa700 1 -- 172.21.15.169:0/1174271885 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498007370 con 0x7f14a0105ce0 2024-04-25T10:51:21.938 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.934+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a01a2c80 con 0x7f14a0105ce0 2024-04-25T10:51:21.939 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.935+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a0051190 con 0x7f14a0105ce0 2024-04-25T10:51:21.939 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.937+0000 7f149d7fa700 1 -- 172.21.15.169:0/1174271885 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f149801c070 con 0x7f14a0105ce0 2024-04-25T10:51:21.939 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.937+0000 7f149d7fa700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb80 0x7f148c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:21.939 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.937+0000 7f149d7fa700 1 -- 172.21.15.169:0/1174271885 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f14980de2c0 con 0x7f14a0105ce0 2024-04-25T10:51:21.939 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.937+0000 7f149f7fe700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb80 0x7f148c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:21.939 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.938+0000 7f149f7fe700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb80 0x7f148c0b2090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14a0105240 tx=0x7f1490009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:21.942 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:21.941+0000 7f149d7fa700 1 -- 172.21.15.169:0/1174271885 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1498023070 con 0x7f14a0105ce0 2024-04-25T10:51:22.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:22.148+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f14a0063ad0 con 0x7f148c0afb80 2024-04-25T10:51:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:22 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:51:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:22 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:51:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:22 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:22 smithi169 ceph-mon[33049]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T10:51:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:22 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:51:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:22 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:51:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:22 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:22 smithi112 ceph-mon[32799]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T10:51:22.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:22 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:51:22.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:22 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:51:22.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:22 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:22.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:22 smithi019 ceph-mon[29209]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T10:51:23.710 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:23 smithi112 ceph-mon[32799]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:23.710 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:23 smithi112 ceph-mon[32799]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:51:23.727 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:23 smithi169 ceph-mon[33049]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:23.727 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:23 smithi169 ceph-mon[33049]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:51:23.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:23 smithi019 ceph-mon[29209]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:23.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:23 smithi019 ceph-mon[29209]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:51:25.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:25 smithi169 ceph-mon[33049]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-25T10:51:25.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:25 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/709737364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]: dispatch 2024-04-25T10:51:25.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:25 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]: dispatch 2024-04-25T10:51:25.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:25 smithi169 ceph-mon[33049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]': finished 2024-04-25T10:51:25.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:25 smithi169 ceph-mon[33049]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T10:51:25.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:25 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:25 smithi112 ceph-mon[32799]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-25T10:51:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:25 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/709737364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]: dispatch 2024-04-25T10:51:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:25 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]: dispatch 2024-04-25T10:51:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:25 smithi112 ceph-mon[32799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]': finished 2024-04-25T10:51:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:25 smithi112 ceph-mon[32799]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T10:51:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:25 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:25 smithi019 ceph-mon[29209]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-25T10:51:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:25 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/709737364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]: dispatch 2024-04-25T10:51:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:25 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]: dispatch 2024-04-25T10:51:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:25 smithi019 ceph-mon[29209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9479e8e3-d6cf-4826-8411-9c6801ec4fd3"}]': finished 2024-04-25T10:51:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:25 smithi019 ceph-mon[29209]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T10:51:25.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:25 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:26.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:26 smithi169 ceph-mon[33049]: from='client.? 172.21.15.169:0/3994313746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:51:26.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:26 smithi112 ceph-mon[32799]: from='client.? 172.21.15.169:0/3994313746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:51:26.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:26 smithi019 ceph-mon[29209]: from='client.? 172.21.15.169:0/3994313746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:51:27.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:27 smithi169 ceph-mon[33049]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T10:51:27.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:27 smithi112 ceph-mon[32799]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T10:51:27.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:27 smithi019 ceph-mon[29209]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T10:51:29.724 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:29 smithi169 ceph-mon[33049]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:51:29.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:29 smithi112 ceph-mon[32799]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:51:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:29 smithi019 ceph-mon[29209]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T10:51:31.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:31 smithi169 ceph-mon[33049]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:51:31.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:31 smithi112 ceph-mon[32799]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:51:31.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:31 smithi019 ceph-mon[29209]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:51:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:33 smithi169 ceph-mon[33049]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:51:33.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T10:51:33.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:33.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:33 smithi169 ceph-mon[33049]: Deploying daemon osd.11 on smithi169 2024-04-25T10:51:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:33 smithi112 ceph-mon[32799]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:51:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T10:51:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:33.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:33 smithi112 ceph-mon[32799]: Deploying daemon osd.11 on smithi169 2024-04-25T10:51:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:33 smithi019 ceph-mon[29209]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T10:51:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T10:51:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:33 smithi019 ceph-mon[29209]: Deploying daemon osd.11 on smithi169 2024-04-25T10:51:35.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:35 smithi169 ceph-mon[33049]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:35.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:35 smithi112 ceph-mon[32799]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:35.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:35 smithi019 ceph-mon[29209]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:37.634 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:37 smithi169 ceph-mon[33049]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:37.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:37 smithi112 ceph-mon[32799]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:37.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:37 smithi019 ceph-mon[29209]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:39.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:39 smithi169 ceph-mon[33049]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:39.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:39.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:39.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:39 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:39 smithi112 ceph-mon[32799]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:39 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:39 smithi019 ceph-mon[29209]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:39 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:41.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:41 smithi169 ceph-mon[33049]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:41.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:41 smithi112 ceph-mon[32799]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:41.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:41 smithi019 ceph-mon[29209]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:42.176 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 11 on host 'smithi169' 2024-04-25T10:51:42.176 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.175+0000 7f149d7fa700 1 -- 172.21.15.169:0/1174271885 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f14a0063ad0 con 0x7f148c0afb80 2024-04-25T10:51:42.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb80 msgr2=0x7f148c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:42.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb80 0x7f148c0b2090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14a0105240 tx=0x7f1490009190 comp rx=0 tx=0).stop 2024-04-25T10:51:42.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105ce0 msgr2=0x7f14a01a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:42.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105ce0 0x7f14a01a1da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f149800b970 tx=0x7f149800bc80 comp rx=0 tx=0).stop 2024-04-25T10:51:42.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 shutdown_connections 2024-04-25T10:51:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb80 0x7f148c0b2090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a0105ce0 0x7f14a01a1da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14a0104210 0x7f14a019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.177+0000 7f14a5ce6700 1 --2- 172.21.15.169:0/1174271885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a0105340 0x7f14a019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.178+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 >> 172.21.15.169:0/1174271885 conn(0x7f14a00ff870 msgr2=0x7f14a0108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.178+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 shutdown_connections 2024-04-25T10:51:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:42.178+0000 7f14a5ce6700 1 -- 172.21.15.169:0/1174271885 wait complete. 2024-04-25T10:51:42.472 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.472 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.472 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:42.472 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:42.472 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.472 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:51:42.472 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:51:42.473 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.473 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.473 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:42 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:51:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:42 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:42.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:42 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:42.861 DEBUG:teuthology.orchestra.run.smithi169:osd.11> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T10:51:42.864 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T10:51:42.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T10:51:43.123 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:51:43.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-mon[33049]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:43.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-mon[33049]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T10:51:43.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-mon[33049]: from='osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:51:43.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-mon[33049]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T10:51:43.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:43.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-mon[33049]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:51:43.351 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T10:51:43.335+0000 7fada503f700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T10:51:43.352 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 10:51:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T10:51:43.350+0000 7fada0df4700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:51:43.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:43 smithi019 ceph-mon[29209]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:43.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:43 smithi019 ceph-mon[29209]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T10:51:43.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:43 smithi019 ceph-mon[29209]: from='osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:51:43.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:43 smithi019 ceph-mon[29209]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T10:51:43.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:43 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:43.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:43 smithi019 ceph-mon[29209]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:51:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:43 smithi112 ceph-mon[32799]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:43 smithi112 ceph-mon[32799]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T10:51:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:43 smithi112 ceph-mon[32799]: from='osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:51:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:43 smithi112 ceph-mon[32799]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T10:51:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:43 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:43 smithi112 ceph-mon[32799]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T10:51:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:44 smithi019 ceph-mon[29209]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:51:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:44 smithi019 ceph-mon[29209]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T10:51:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:44 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:44.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:44 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:44.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:44 smithi169 ceph-mon[33049]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:51:44.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:44 smithi169 ceph-mon[33049]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T10:51:44.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:44 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:44.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:44 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 -- 172.21.15.19:0/2213774696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14104be0 msgr2=0x7faa1410b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 --2- 172.21.15.19:0/2213774696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14104be0 0x7faa1410b490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faa04009990 tx=0x7faa04009ca0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 -- 172.21.15.19:0/2213774696 shutdown_connections 2024-04-25T10:51:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 --2- 172.21.15.19:0/2213774696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14104be0 0x7faa1410b490 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 --2- 172.21.15.19:0/2213774696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa14103110 0x7faa14103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 --2- 172.21.15.19:0/2213774696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa14104240 0x7faa14104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 -- 172.21.15.19:0/2213774696 >> 172.21.15.19:0/2213774696 conn(0x7faa140fe750 msgr2=0x7faa14100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 -- 172.21.15.19:0/2213774696 shutdown_connections 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.769+0000 7faa1bb40700 1 -- 172.21.15.19:0/2213774696 wait complete. 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 Processor -- start 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 -- start start 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14103110 0x7faa1419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa14104240 0x7faa1419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa14104be0 0x7faa141a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa1410ec50 con 0x7faa14103110 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa1410ead0 con 0x7faa14104be0 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa1bb40700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa1410edd0 con 0x7faa14104240 2024-04-25T10:51:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa198dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14103110 0x7faa1419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa198dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14103110 0x7faa1419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34848/0 (socket says 172.21.15.19:34848) 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa198dc700 1 -- 172.21.15.19:0/3094888833 learned_addr learned my addr 172.21.15.19:0/3094888833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.770+0000 7faa190db700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa14104240 0x7faa1419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa1a0dd700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa14104be0 0x7faa141a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa198dc700 1 -- 172.21.15.19:0/3094888833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa14104240 msgr2=0x7faa1419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa198dc700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa14104240 0x7faa1419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa198dc700 1 -- 172.21.15.19:0/3094888833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa14104be0 msgr2=0x7faa141a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa198dc700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa14104be0 0x7faa141a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa198dc700 1 -- 172.21.15.19:0/3094888833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa141a23f0 con 0x7faa14103110 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa198dc700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14103110 0x7faa1419a890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007ee0 tx=0x7faa0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa12ffd700 1 -- 172.21.15.19:0/3094888833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00dbd0 con 0x7faa14103110 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa12ffd700 1 -- 172.21.15.19:0/3094888833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa0c00dd70 con 0x7faa14103110 2024-04-25T10:51:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa141a26e0 con 0x7faa14103110 2024-04-25T10:51:44.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa12ffd700 1 -- 172.21.15.19:0/3094888833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0047b0 con 0x7faa14103110 2024-04-25T10:51:44.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.771+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa141a2ca0 con 0x7faa14103110 2024-04-25T10:51:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.772+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa14051190 con 0x7faa14103110 2024-04-25T10:51:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.774+0000 7faa12ffd700 1 -- 172.21.15.19:0/3094888833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7faa0c01a020 con 0x7faa14103110 2024-04-25T10:51:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.774+0000 7faa12ffd700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9fc0afb80 0x7fa9fc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.774+0000 7faa190db700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9fc0afb80 0x7fa9fc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.774+0000 7faa12ffd700 1 -- 172.21.15.19:0/3094888833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7faa0c0dde80 con 0x7faa14103110 2024-04-25T10:51:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.775+0000 7faa190db700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9fc0afb80 0x7fa9fc0b2090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa1419b940 tx=0x7faa08009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.779+0000 7faa12ffd700 1 -- 172.21.15.19:0/3094888833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa0c010070 con 0x7faa14103110 2024-04-25T10:51:44.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:44 smithi112 ceph-mon[32799]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T10:51:44.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:44 smithi112 ceph-mon[32799]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T10:51:44.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:44 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:44.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:44 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.989+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7faa141a30c0 con 0x7faa14103110 2024-04-25T10:51:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.989+0000 7faa12ffd700 1 -- 172.21.15.19:0/3094888833 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7faa0c0a6f00 con 0x7faa14103110 2024-04-25T10:51:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:51:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9fc0afb80 msgr2=0x7fa9fc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9fc0afb80 0x7fa9fc0b2090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa1419b940 tx=0x7faa08009190 comp rx=0 tx=0).stop 2024-04-25T10:51:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14103110 msgr2=0x7faa1419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14103110 0x7faa1419a890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007ee0 tx=0x7faa0c00d3b0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 shutdown_connections 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9fc0afb80 0x7fa9fc0b2090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa14104be0 0x7faa141a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14103110 0x7faa1419a890 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 --2- 172.21.15.19:0/3094888833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa14104240 0x7faa1419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 >> 172.21.15.19:0/3094888833 conn(0x7faa140fe750 msgr2=0x7faa14100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.991+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 shutdown_connections 2024-04-25T10:51:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:44.992+0000 7faa1bb40700 1 -- 172.21.15.19:0/3094888833 wait complete. 2024-04-25T10:51:45.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:45 smithi019 ceph-mon[29209]: purged_snaps scrub starts 2024-04-25T10:51:45.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:45 smithi019 ceph-mon[29209]: purged_snaps scrub ok 2024-04-25T10:51:45.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:45 smithi019 ceph-mon[29209]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:45.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:45 smithi019 ceph-mon[29209]: osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] boot 2024-04-25T10:51:45.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:45 smithi019 ceph-mon[29209]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T10:51:45.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:45.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:45 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3094888833' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:51:45.606 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1714042304,"num_in_osds":12,"osd_in_since":1714042284,"num_remapped_pgs":0} 2024-04-25T10:51:45.607 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T10:51:45.687 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:45 smithi169 ceph-mon[33049]: purged_snaps scrub starts 2024-04-25T10:51:45.687 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:45 smithi169 ceph-mon[33049]: purged_snaps scrub ok 2024-04-25T10:51:45.687 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:45 smithi169 ceph-mon[33049]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:45.687 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:45 smithi169 ceph-mon[33049]: osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] boot 2024-04-25T10:51:45.687 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:45 smithi169 ceph-mon[33049]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T10:51:45.687 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:45.687 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:45 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3094888833' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:51:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:45 smithi112 ceph-mon[32799]: purged_snaps scrub starts 2024-04-25T10:51:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:45 smithi112 ceph-mon[32799]: purged_snaps scrub ok 2024-04-25T10:51:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:45 smithi112 ceph-mon[32799]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:51:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:45 smithi112 ceph-mon[32799]: osd.11 [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] boot 2024-04-25T10:51:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:45 smithi112 ceph-mon[32799]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T10:51:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:51:45.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:45 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3094888833' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:51:45.866 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:51:46.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T10:51:46.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:46.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:46.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:46 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:46.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:46 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T10:51:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:51:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:46 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 -- 172.21.15.19:0/2199753536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44103390 msgr2=0x7fce441037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 --2- 172.21.15.19:0/2199753536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44103390 0x7fce441037a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce400099c0 tx=0x7fce40009cd0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 -- 172.21.15.19:0/2199753536 shutdown_connections 2024-04-25T10:51:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 --2- 172.21.15.19:0/2199753536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44108610 0x7fce4410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 --2- 172.21.15.19:0/2199753536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44103390 0x7fce441037a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 --2- 172.21.15.19:0/2199753536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce44107cc0 0x7fce441080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 -- 172.21.15.19:0/2199753536 >> 172.21.15.19:0/2199753536 conn(0x7fce440fec90 msgr2=0x7fce441010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 -- 172.21.15.19:0/2199753536 shutdown_connections 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 -- 172.21.15.19:0/2199753536 wait complete. 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 Processor -- start 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.477+0000 7fce4c40b700 1 -- start start 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4c40b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44103390 0x7fce4419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4c40b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce44107cc0 0x7fce4419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4c40b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44108610 0x7fce4419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4c40b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce441132c0 con 0x7fce44108610 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4c40b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce44113140 con 0x7fce44103390 2024-04-25T10:51:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4c40b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce44113440 con 0x7fce44107cc0 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44108610 0x7fce4419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44108610 0x7fce4419f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34852/0 (socket says 172.21.15.19:34852) 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 -- 172.21.15.19:0/15644594 learned_addr learned my addr 172.21.15.19:0/15644594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce499a6700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce44107cc0 0x7fce4419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a1a7700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44103390 0x7fce4419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 -- 172.21.15.19:0/15644594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce44107cc0 msgr2=0x7fce4419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce44107cc0 0x7fce4419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 -- 172.21.15.19:0/15644594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44103390 msgr2=0x7fce4419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44103390 0x7fce4419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 -- 172.21.15.19:0/15644594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4419fbf0 con 0x7fce44108610 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.478+0000 7fce4a9a8700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44108610 0x7fce4419f550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b970 tx=0x7fce3c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.479+0000 7fce377fe700 1 -- 172.21.15.19:0/15644594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00d860 con 0x7fce44108610 2024-04-25T10:51:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.479+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4419fee0 con 0x7fce44108610 2024-04-25T10:51:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.479+0000 7fce377fe700 1 -- 172.21.15.19:0/15644594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c00e430 con 0x7fce44108610 2024-04-25T10:51:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.479+0000 7fce377fe700 1 -- 172.21.15.19:0/15644594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c017660 con 0x7fce44108610 2024-04-25T10:51:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.479+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce441a7800 con 0x7fce44108610 2024-04-25T10:51:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.480+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce44051190 con 0x7fce44108610 2024-04-25T10:51:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.482+0000 7fce377fe700 1 -- 172.21.15.19:0/15644594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fce3c00da00 con 0x7fce44108610 2024-04-25T10:51:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.483+0000 7fce377fe700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce300afb80 0x7fce300b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.483+0000 7fce4a1a7700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce300afb80 0x7fce300b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.483+0000 7fce377fe700 1 -- 172.21.15.19:0/15644594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fce3c0de620 con 0x7fce44108610 2024-04-25T10:51:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.483+0000 7fce4a1a7700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce300afb80 0x7fce300b2090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fce40009990 tx=0x7fce40000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.486+0000 7fce377fe700 1 -- 172.21.15.19:0/15644594 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce3c052ff0 con 0x7fce44108610 2024-04-25T10:51:47.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:47 smithi019 ceph-mon[29209]: Detected new or changed devices on smithi169 2024-04-25T10:51:47.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:47 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T10:51:47.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:47 smithi019 ceph-mon[29209]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.695+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce4402e510 con 0x7fce44108610 2024-04-25T10:51:47.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.696+0000 7fce377fe700 1 -- 172.21.15.19:0/15644594 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18463 (secure 0 0 0) 0x7fce3c013030 con 0x7fce44108610 2024-04-25T10:51:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:51:47.699 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":73,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:51:45.338653+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.698+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce300afb80 msgr2=0x7fce300b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.698+0000 7fce4c40b700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce300afb80 0x7fce300b2090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fce40009990 tx=0x7fce40000bc0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.698+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44108610 msgr2=0x7fce4419f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.698+0000 7fce4c40b700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44108610 0x7fce4419f550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b970 tx=0x7fce3c00bc80 comp rx=0 tx=0).stop 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 shutdown_connections 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce300afb80 0x7fce300b2090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce44108610 0x7fce4419f550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44103390 0x7fce4419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 --2- 172.21.15.19:0/15644594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce44107cc0 0x7fce4419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:47.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 >> 172.21.15.19:0/15644594 conn(0x7fce440fec90 msgr2=0x7fce44101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:47.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 shutdown_connections 2024-04-25T10:51:47.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:47.699+0000 7fce4c40b700 1 -- 172.21.15.19:0/15644594 wait complete. 2024-04-25T10:51:47.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:47 smithi169 ceph-mon[33049]: Detected new or changed devices on smithi169 2024-04-25T10:51:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:47 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T10:51:47.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:47 smithi169 ceph-mon[33049]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:47 smithi112 ceph-mon[32799]: Detected new or changed devices on smithi169 2024-04-25T10:51:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:47 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T10:51:47.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:47 smithi112 ceph-mon[32799]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:48.306 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:51:48.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T10:51:48.555 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:51:48.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:48 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/15644594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:51:48.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:48 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/15644594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:51:48.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:48 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/15644594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:51:49.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:49 smithi019 ceph-mon[29209]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:49.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:49 smithi169 ceph-mon[33049]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:49.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:49 smithi112 ceph-mon[32799]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 -- 172.21.15.19:0/1548800885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0107c90 msgr2=0x7fc5b01080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 --2- 172.21.15.19:0/1548800885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0107c90 0x7fc5b01080a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40099c0 tx=0x7fc5a4009cd0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 -- 172.21.15.19:0/1548800885 shutdown_connections 2024-04-25T10:51:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 --2- 172.21.15.19:0/1548800885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b01085e0 0x7fc5b010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 --2- 172.21.15.19:0/1548800885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b0103360 0x7fc5b0103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 --2- 172.21.15.19:0/1548800885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0107c90 0x7fc5b01080a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 -- 172.21.15.19:0/1548800885 >> 172.21.15.19:0/1548800885 conn(0x7fc5b00fecb0 msgr2=0x7fc5b01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.134+0000 7fc5b611d700 1 -- 172.21.15.19:0/1548800885 shutdown_connections 2024-04-25T10:51:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.135+0000 7fc5b611d700 1 -- 172.21.15.19:0/1548800885 wait complete. 2024-04-25T10:51:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.135+0000 7fc5b611d700 1 Processor -- start 2024-04-25T10:51:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5b611d700 1 -- start start 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5b611d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0103360 0x7fc5b019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5b611d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b0107c90 0x7fc5b019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5b611d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b01085e0 0x7fc5b019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5b611d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01131f0 con 0x7fc5b0103360 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5b611d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0113070 con 0x7fc5b01085e0 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5b611d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0113370 con 0x7fc5b0107c90 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0103360 0x7fc5b019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0103360 0x7fc5b019bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34872/0 (socket says 172.21.15.19:34872) 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 -- 172.21.15.19:0/2513252489 learned_addr learned my addr 172.21.15.19:0/2513252489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5aeffd700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b0107c90 0x7fc5b019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5affff700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b01085e0 0x7fc5b019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 -- 172.21.15.19:0/2513252489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b0107c90 msgr2=0x7fc5b019c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b0107c90 0x7fc5b019c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 -- 172.21.15.19:0/2513252489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b01085e0 msgr2=0x7fc5b019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b01085e0 0x7fc5b019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.136+0000 7fc5af7fe700 1 -- 172.21.15.19:0/2513252489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b01a6ea0 con 0x7fc5b0103360 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.137+0000 7fc5af7fe700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0103360 0x7fc5b019bb30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000d950 tx=0x7fc5a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.137+0000 7fc5acff9700 1 -- 172.21.15.19:0/2513252489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0016070 con 0x7fc5b0103360 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.137+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b01a7190 con 0x7fc5b0103360 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.137+0000 7fc5acff9700 1 -- 172.21.15.19:0/2513252489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a000fd40 con 0x7fc5b0103360 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.137+0000 7fc5acff9700 1 -- 172.21.15.19:0/2513252489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e3b0 con 0x7fc5b0103360 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.137+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a76f0 con 0x7fc5b0103360 2024-04-25T10:51:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.138+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b0051190 con 0x7fc5b0103360 2024-04-25T10:51:50.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.139+0000 7fc5acff9700 1 -- 172.21.15.19:0/2513252489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc5a0019070 con 0x7fc5b0103360 2024-04-25T10:51:50.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.140+0000 7fc5acff9700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc5980afc00 0x7fc5980b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:50.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.140+0000 7fc5aeffd700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc5980afc00 0x7fc5980b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:50.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.140+0000 7fc5acff9700 1 -- 172.21.15.19:0/2513252489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc5a00de540 con 0x7fc5b0103360 2024-04-25T10:51:50.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.140+0000 7fc5aeffd700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc5980afc00 0x7fc5980b2110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4006010 tx=0x7fc5a4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.144+0000 7fc5acff9700 1 -- 172.21.15.19:0/2513252489 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5a00a75c0 con 0x7fc5b0103360 2024-04-25T10:51:50.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.348+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5b002e630 con 0x7fc5b0103360 2024-04-25T10:51:50.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.348+0000 7fc5acff9700 1 -- 172.21.15.19:0/2513252489 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fc5a00a6ec0 con 0x7fc5b0103360 2024-04-25T10:51:50.350 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:51:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc5980afc00 msgr2=0x7fc5980b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc5980afc00 0x7fc5980b2110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4006010 tx=0x7fc5a4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0103360 msgr2=0x7fc5b019bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0103360 0x7fc5b019bb30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000d950 tx=0x7fc5a000dc60 comp rx=0 tx=0).stop 2024-04-25T10:51:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 shutdown_connections 2024-04-25T10:51:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc5980afc00 0x7fc5980b2110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b01085e0 0x7fc5b019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5b0103360 0x7fc5b019bb30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 --2- 172.21.15.19:0/2513252489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b0107c90 0x7fc5b019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 >> 172.21.15.19:0/2513252489 conn(0x7fc5b00fecb0 msgr2=0x7fc5b0107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 shutdown_connections 2024-04-25T10:51:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:50.350+0000 7fc5b611d700 1 -- 172.21.15.19:0/2513252489 wait complete. 2024-04-25T10:51:50.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:50 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2513252489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:51:50.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:50 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2513252489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:51:50.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:50 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2513252489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:51:51.050 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-04-25T10:51:51.050 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.b on smithi112 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.e on smithi112 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.h on smithi112 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.k on smithi112 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.c on smithi169 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.f on smithi169 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.i on smithi169 2024-04-25T10:51:51.051 INFO:tasks.cephadm:Adding mds.l on smithi169 2024-04-25T10:51:51.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l' 2024-04-25T10:51:51.303 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:51:51.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:51 smithi169 ceph-mon[33049]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:51.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:51 smithi112 ceph-mon[32799]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:51.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:51 smithi019 ceph-mon[29209]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:52.722 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 -- 172.21.15.169:0/1552232481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c8105880 msgr2=0x7fc9c8105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1552232481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c8105880 0x7fc9c8105c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009990 tx=0x7fc9b8009ca0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 -- 172.21.15.169:0/1552232481 shutdown_connections 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1552232481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9c81048b0 0x7fc9c810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1552232481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c8105880 0x7fc9c8105c90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1552232481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c8103e30 0x7fc9c8104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 -- 172.21.15.169:0/1552232481 >> 172.21.15.169:0/1552232481 conn(0x7fc9c80ff8a0 msgr2=0x7fc9c8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 -- 172.21.15.169:0/1552232481 shutdown_connections 2024-04-25T10:51:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.722+0000 7fc9d0555700 1 -- 172.21.15.169:0/1552232481 wait complete. 2024-04-25T10:51:52.729 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 Processor -- start 2024-04-25T10:51:52.729 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 -- start start 2024-04-25T10:51:52.729 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c8103e30 0x7fc9c819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:52.729 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c81048b0 0x7fc9c819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:52.729 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9c8105880 0x7fc9c81a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:52.729 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c810fe90 con 0x7fc9c8105880 2024-04-25T10:51:52.729 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c810fd10 con 0x7fc9c8103e30 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9d0555700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c8110010 con 0x7fc9c81048b0 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c81048b0 0x7fc9c819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c81048b0 0x7fc9c819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47364/0 (socket says 172.21.15.169:47364) 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 -- 172.21.15.169:0/1132162031 learned_addr learned my addr 172.21.15.169:0/1132162031 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9ce2f1700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c8103e30 0x7fc9c819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9ceaf2700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9c8105880 0x7fc9c81a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 -- 172.21.15.169:0/1132162031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c8103e30 msgr2=0x7fc9c819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:52.730 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c8103e30 0x7fc9c819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 -- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9c8105880 msgr2=0x7fc9c81a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9c8105880 0x7fc9c81a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.723+0000 7fc9cdaf0700 1 -- 172.21.15.169:0/1132162031 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c81a25c0 con 0x7fc9c81048b0 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.724+0000 7fc9cdaf0700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c81048b0 0x7fc9c819afa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400b530 tx=0x7fc9c400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.724+0000 7fc9bf7fe700 1 -- 172.21.15.169:0/1132162031 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4015020 con 0x7fc9c81048b0 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.724+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c81a28b0 con 0x7fc9c81048b0 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.724+0000 7fc9bf7fe700 1 -- 172.21.15.169:0/1132162031 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c4011ce0 con 0x7fc9c81048b0 2024-04-25T10:51:52.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.724+0000 7fc9bf7fe700 1 -- 172.21.15.169:0/1132162031 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4010e40 con 0x7fc9c81048b0 2024-04-25T10:51:52.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.724+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c81a2e70 con 0x7fc9c81048b0 2024-04-25T10:51:52.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.725+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c8051190 con 0x7fc9c81048b0 2024-04-25T10:51:52.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.726+0000 7fc9bf7fe700 1 -- 172.21.15.169:0/1132162031 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc9c4018020 con 0x7fc9c81048b0 2024-04-25T10:51:52.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.727+0000 7fc9bf7fe700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9b40afb80 0x7fc9b40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:52.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.727+0000 7fc9bf7fe700 1 -- 172.21.15.169:0/1132162031 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc9c40de560 con 0x7fc9c81048b0 2024-04-25T10:51:52.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.727+0000 7fc9ce2f1700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9b40afb80 0x7fc9b40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:52.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.728+0000 7fc9ce2f1700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9b40afb80 0x7fc9b40b2090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80038a0 tx=0x7fc9b801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:52.733 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.731+0000 7fc9bf7fe700 1 -- 172.21.15.169:0/1132162031 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9c401f020 con 0x7fc9c81048b0 2024-04-25T10:51:52.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.937+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}) -- 0x7fc9c8063ad0 con 0x7fc9b40afb80 2024-04-25T10:51:52.946 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.946+0000 7fc9bf7fe700 1 -- 172.21.15.169:0/1132162031 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc9c8063ad0 con 0x7fc9b40afb80 2024-04-25T10:51:52.946 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mds.all update... 2024-04-25T10:51:52.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9b40afb80 msgr2=0x7fc9b40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:52.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9b40afb80 0x7fc9b40b2090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80038a0 tx=0x7fc9b801a040 comp rx=0 tx=0).stop 2024-04-25T10:51:52.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c81048b0 msgr2=0x7fc9c819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c81048b0 0x7fc9c819afa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400b530 tx=0x7fc9c400b840 comp rx=0 tx=0).stop 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 shutdown_connections 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9b40afb80 0x7fc9b40b2090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9c8105880 0x7fc9c81a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c8103e30 0x7fc9c819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 --2- 172.21.15.169:0/1132162031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c81048b0 0x7fc9c819afa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 >> 172.21.15.169:0/1132162031 conn(0x7fc9c80ff8a0 msgr2=0x7fc9c81085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 shutdown_connections 2024-04-25T10:51:52.949 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:51:52.948+0000 7fc9d0555700 1 -- 172.21.15.169:0/1132162031 wait complete. 2024-04-25T10:51:53.546 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:53 smithi169 ceph-mon[33049]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:53.546 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:53.546 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:53 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:53.575 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T10:51:53.578 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T10:51:53.620 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T10:51:53.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:53 smithi019 ceph-mon[29209]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:53.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:53.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:53 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:53.642 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T10:51:53.685 DEBUG:teuthology.orchestra.run.smithi112:mds.b> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T10:51:53.688 DEBUG:teuthology.orchestra.run.smithi112:mds.e> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T10:51:53.708 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:53 smithi112 ceph-mon[32799]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:53.708 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:53.708 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:53 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:51:53.709 DEBUG:teuthology.orchestra.run.smithi112:mds.h> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T10:51:53.752 DEBUG:teuthology.orchestra.run.smithi112:mds.k> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T10:51:53.795 DEBUG:teuthology.orchestra.run.smithi169:mds.c> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T10:51:53.797 DEBUG:teuthology.orchestra.run.smithi169:mds.f> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T10:51:53.840 DEBUG:teuthology.orchestra.run.smithi169:mds.i> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T10:51:53.883 DEBUG:teuthology.orchestra.run.smithi169:mds.l> sudo journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T10:51:53.926 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T10:51:53.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:51:54.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 -- 172.21.15.19:0/1822858622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241033c0 msgr2=0x7f0c241037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1822858622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241033c0 0x7f0c241037d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009900 tx=0x7f0c1802a410 comp rx=0 tx=0).stop 2024-04-25T10:51:54.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 -- 172.21.15.19:0/1822858622 shutdown_connections 2024-04-25T10:51:54.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1822858622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c241080c0 0x7f0c2410a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1822858622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241033c0 0x7f0c241037d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1822858622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c24102180 0x7f0c24102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 -- 172.21.15.19:0/1822858622 >> 172.21.15.19:0/1822858622 conn(0x7f0c240fd3b0 msgr2=0x7f0c240ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 -- 172.21.15.19:0/1822858622 shutdown_connections 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.042+0000 7f0c28a92640 1 -- 172.21.15.19:0/1822858622 wait complete. 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 Processor -- start 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 -- start start 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c24102180 0x7f0c24197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c241033c0 0x7f0c24198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241080c0 0x7f0c2419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410d410 con 0x7f0c241080c0 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410d290 con 0x7f0c241033c0 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c28a92640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410d590 con 0x7f0c24102180 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c22d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241080c0 0x7f0c2419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c22d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241080c0 0x7f0c2419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58332/0 (socket says 172.21.15.19:58332) 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c22d9d640 1 -- 172.21.15.19:0/1843984833 learned_addr learned my addr 172.21.15.19:0/1843984833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c2259c640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c24102180 0x7f0c24197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c21d9b640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c241033c0 0x7f0c24198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c22d9d640 1 -- 172.21.15.19:0/1843984833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c24102180 msgr2=0x7f0c24197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c22d9d640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c24102180 0x7f0c24197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c22d9d640 1 -- 172.21.15.19:0/1843984833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c241033c0 msgr2=0x7f0c24198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.043+0000 7f0c22d9d640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c241033c0 0x7f0c24198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.044+0000 7f0c22d9d640 1 -- 172.21.15.19:0/1843984833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2419fb30 con 0x7f0c241080c0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.044+0000 7f0c22d9d640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241080c0 0x7f0c2419f4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00eaa0 tx=0x7f0c1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.044+0000 7f0c137fe640 1 -- 172.21.15.19:0/1843984833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00cdb0 con 0x7f0c241080c0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.044+0000 7f0c137fe640 1 -- 172.21.15.19:0/1843984833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c1c005020 con 0x7f0c241080c0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.044+0000 7f0c137fe640 1 -- 172.21.15.19:0/1843984833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0116d0 con 0x7f0c241080c0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.044+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2419fe20 con 0x7f0c241080c0 2024-04-25T10:51:54.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.044+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241a0280 con 0x7f0c241080c0 2024-04-25T10:51:54.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.045+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2406a1c0 con 0x7f0c241080c0 2024-04-25T10:51:54.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.047+0000 7f0c137fe640 1 -- 172.21.15.19:0/1843984833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0c1c0051d0 con 0x7f0c241080c0 2024-04-25T10:51:54.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.048+0000 7f0c137fe640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0c0c0afb70 0x7f0c0c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.048+0000 7f0c2259c640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0c0c0afb70 0x7f0c0c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.048+0000 7f0c137fe640 1 -- 172.21.15.19:0/1843984833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f0c1c0deb90 con 0x7f0c241080c0 2024-04-25T10:51:54.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.048+0000 7f0c2259c640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0c0c0afb70 0x7f0c0c0b2080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c14005fd0 tx=0x7f0c14010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:54.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.052+0000 7f0c137fe640 1 -- 172.21.15.19:0/1843984833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c2406a1c0 con 0x7f0c241080c0 2024-04-25T10:51:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.244+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c2402dc10 con 0x7f0c241080c0 2024-04-25T10:51:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.245+0000 7f0c137fe640 1 -- 172.21.15.19:0/1843984833 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18463 (secure 0 0 0) 0x7f0c1c016030 con 0x7f0c241080c0 2024-04-25T10:51:54.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:51:54.248 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":73,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:51:45.338653+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0c0c0afb70 msgr2=0x7f0c0c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0c0c0afb70 0x7f0c0c0b2080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c14005fd0 tx=0x7f0c14010040 comp rx=0 tx=0).stop 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241080c0 msgr2=0x7f0c2419f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241080c0 0x7f0c2419f4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00eaa0 tx=0x7f0c1c00ef60 comp rx=0 tx=0).stop 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 shutdown_connections 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0c0c0afb70 0x7f0c0c0b2080 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c241080c0 0x7f0c2419f4c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c241033c0 0x7f0c24198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 --2- 172.21.15.19:0/1843984833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c24102180 0x7f0c24197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 >> 172.21.15.19:0/1843984833 conn(0x7f0c240fd3b0 msgr2=0x7f0c241040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 shutdown_connections 2024-04-25T10:51:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.248+0000 7f0c28a92640 1 -- 172.21.15.19:0/1843984833 wait complete. 2024-04-25T10:51:54.261 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:51:54.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:51:54.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 -- 172.21.15.19:0/2599383752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634105090 msgr2=0x7f76341054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 --2- 172.21.15.19:0/2599383752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634105090 0x7f76341054a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f762c009900 tx=0x7f762c02a410 comp rx=0 tx=0).stop 2024-04-25T10:51:54.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 -- 172.21.15.19:0/2599383752 shutdown_connections 2024-04-25T10:51:54.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 --2- 172.21.15.19:0/2599383752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7634102240 0x7f763410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 --2- 172.21.15.19:0/2599383752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7634101740 0x7f7634101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 --2- 172.21.15.19:0/2599383752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634105090 0x7f76341054a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 -- 172.21.15.19:0/2599383752 >> 172.21.15.19:0/2599383752 conn(0x7f76340fd340 msgr2=0x7f76340ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 -- 172.21.15.19:0/2599383752 shutdown_connections 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.375+0000 7f7639451640 1 -- 172.21.15.19:0/2599383752 wait complete. 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 Processor -- start 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 -- start start 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634101740 0x7f763419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7634102240 0x7f763419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7634105090 0x7f76341a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f763410f600 con 0x7f7634101740 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f763410f480 con 0x7f7634105090 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7639451640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f763410f780 con 0x7f7634102240 2024-04-25T10:51:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634101740 0x7f763419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634101740 0x7f763419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58358/0 (socket says 172.21.15.19:58358) 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 -- 172.21.15.19:0/3994362191 learned_addr learned my addr 172.21.15.19:0/3994362191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f76337fe640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7634105090 0x7f76341a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f76327fc640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7634102240 0x7f763419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 -- 172.21.15.19:0/3994362191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7634102240 msgr2=0x7f763419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7634102240 0x7f763419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 -- 172.21.15.19:0/3994362191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7634105090 msgr2=0x7f76341a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7634105090 0x7f76341a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f7632ffd640 1 -- 172.21.15.19:0/3994362191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76341a1da0 con 0x7f7634101740 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.376+0000 7f76337fe640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7634105090 0x7f76341a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.377+0000 7f7632ffd640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634101740 0x7f763419a260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f762c006010 tx=0x7f762c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.377+0000 7f762bfff640 1 -- 172.21.15.19:0/3994362191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c03f030 con 0x7f7634101740 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.377+0000 7f762bfff640 1 -- 172.21.15.19:0/3994362191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762c031910 con 0x7f7634101740 2024-04-25T10:51:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.377+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76341a2030 con 0x7f7634101740 2024-04-25T10:51:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.377+0000 7f762bfff640 1 -- 172.21.15.19:0/3994362191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c035070 con 0x7f7634101740 2024-04-25T10:51:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.377+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341a2520 con 0x7f7634101740 2024-04-25T10:51:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.378+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7634108a20 con 0x7f7634101740 2024-04-25T10:51:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.379+0000 7f762bfff640 1 -- 172.21.15.19:0/3994362191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f762c042030 con 0x7f7634101740 2024-04-25T10:51:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.380+0000 7f762bfff640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f76200afaf0 0x7f76200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.380+0000 7f76327fc640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f76200afaf0 0x7f76200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.380+0000 7f762bfff640 1 -- 172.21.15.19:0/3994362191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f762c0ff090 con 0x7f7634101740 2024-04-25T10:51:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.380+0000 7f76327fc640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f76200afaf0 0x7f76200b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f763419b300 tx=0x7f761c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:54.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.384+0000 7f762bfff640 1 -- 172.21.15.19:0/3994362191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f762c0c8070 con 0x7f7634101740 2024-04-25T10:51:54.527 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:54.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:54 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1843984833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:51:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.570+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76340637a0 con 0x7f7634101740 2024-04-25T10:51:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.571+0000 7f762bfff640 1 -- 172.21.15.19:0/3994362191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f762c0d00f0 con 0x7f7634101740 2024-04-25T10:51:54.573 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:51:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f76200afaf0 msgr2=0x7f76200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f76200afaf0 0x7f76200b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f763419b300 tx=0x7f761c005f90 comp rx=0 tx=0).stop 2024-04-25T10:51:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634101740 msgr2=0x7f763419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634101740 0x7f763419a260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f762c006010 tx=0x7f762c004270 comp rx=0 tx=0).stop 2024-04-25T10:51:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 shutdown_connections 2024-04-25T10:51:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7634105090 0x7f76341a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7634102240 0x7f763419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f76200afaf0 0x7f76200b2000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 --2- 172.21.15.19:0/3994362191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634101740 0x7f763419a260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 >> 172.21.15.19:0/3994362191 conn(0x7f76340fd340 msgr2=0x7f7634105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 shutdown_connections 2024-04-25T10:51:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.573+0000 7f7639451640 1 -- 172.21.15.19:0/3994362191 wait complete. 2024-04-25T10:51:54.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:51:54.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T10:51:54.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:54.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:54.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:54.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:51:54.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:51:54.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:54.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:54 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1843984833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:51:54.587 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 -- 172.21.15.19:0/388298911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e01029c0 msgr2=0x7f24e01036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 --2- 172.21.15.19:0/388298911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e01029c0 0x7f24e01036c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009900 tx=0x7f24dc02a410 comp rx=0 tx=0).stop 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 -- 172.21.15.19:0/388298911 shutdown_connections 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 --2- 172.21.15.19:0/388298911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24e0108900 0x7f24e010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 --2- 172.21.15.19:0/388298911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24e0103c00 0x7f24e0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 --2- 172.21.15.19:0/388298911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e01029c0 0x7f24e01036c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 -- 172.21.15.19:0/388298911 >> 172.21.15.19:0/388298911 conn(0x7f24e00fdbd0 msgr2=0x7f24e0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 -- 172.21.15.19:0/388298911 shutdown_connections 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.697+0000 7f24e79f3640 1 -- 172.21.15.19:0/388298911 wait complete. 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 Processor -- start 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 -- start start 2024-04-25T10:51:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24e01029c0 0x7f24e00ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e0103c00 0x7f24e00fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24e0108900 0x7f24e00fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010e510 con 0x7f24e0103c00 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010e390 con 0x7f24e0108900 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e79f3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010e690 con 0x7f24e01029c0 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e578f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24e01029c0 0x7f24e00ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e4f8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e0103c00 0x7f24e00fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e578f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24e01029c0 0x7f24e00ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:45332/0 (socket says 172.21.15.19:45332) 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e578f640 1 -- 172.21.15.19:0/998361447 learned_addr learned my addr 172.21.15.19:0/998361447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e5f90640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24e0108900 0x7f24e00fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e4f8e640 1 -- 172.21.15.19:0/998361447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24e01029c0 msgr2=0x7f24e00ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e4f8e640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24e01029c0 0x7f24e00ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e4f8e640 1 -- 172.21.15.19:0/998361447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24e0108900 msgr2=0x7f24e00fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e4f8e640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24e0108900 0x7f24e00fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.698+0000 7f24e4f8e640 1 -- 172.21.15.19:0/998361447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e00fed70 con 0x7f24e0103c00 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.699+0000 7f24e4f8e640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e0103c00 0x7f24e00fe100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24d000d720 tx=0x7f24d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.699+0000 7f24d67fc640 1 -- 172.21.15.19:0/998361447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0017070 con 0x7f24e0103c00 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.699+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e01a42f0 con 0x7f24e0103c00 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.699+0000 7f24d67fc640 1 -- 172.21.15.19:0/998361447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d000de30 con 0x7f24e0103c00 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.699+0000 7f24d67fc640 1 -- 172.21.15.19:0/998361447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d00047a0 con 0x7f24e0103c00 2024-04-25T10:51:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.699+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e01a4760 con 0x7f24e0103c00 2024-04-25T10:51:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.700+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e01a49c0 con 0x7f24e0103c00 2024-04-25T10:51:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.701+0000 7f24d67fc640 1 -- 172.21.15.19:0/998361447 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f24d00099d0 con 0x7f24e0103c00 2024-04-25T10:51:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.702+0000 7f24d67fc640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24cc0afaf0 0x7f24cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.702+0000 7f24e578f640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24cc0afaf0 0x7f24cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.702+0000 7f24d67fc640 1 -- 172.21.15.19:0/998361447 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f24d00de760 con 0x7f24e0103c00 2024-04-25T10:51:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.702+0000 7f24e578f640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24cc0afaf0 0x7f24cc0b2000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24dc000c00 tx=0x7f24dc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.705+0000 7f24d67fc640 1 -- 172.21.15.19:0/998361447 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24d0020020 con 0x7f24e0103c00 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:51:54.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:54.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:54 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1843984833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:51:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.934+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24e01a4b90 con 0x7f24e0103c00 2024-04-25T10:51:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.935+0000 7f24d67fc640 1 -- 172.21.15.19:0/998361447 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f24d00a77e0 con 0x7f24e0103c00 2024-04-25T10:51:54.936 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:51:54.937 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T10:51:54.937 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24cc0afaf0 msgr2=0x7f24cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24cc0afaf0 0x7f24cc0b2000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24dc000c00 tx=0x7f24dc0314a0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e0103c00 msgr2=0x7f24e00fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e0103c00 0x7f24e00fe100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24d000d720 tx=0x7f24d000dbe0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 shutdown_connections 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24e0108900 0x7f24e00fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24cc0afaf0 0x7f24cc0b2000 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e0103c00 0x7f24e00fe100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 --2- 172.21.15.19:0/998361447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24e01029c0 0x7f24e00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 >> 172.21.15.19:0/998361447 conn(0x7f24e00fdbd0 msgr2=0x7f24e0109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 shutdown_connections 2024-04-25T10:51:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:54.937+0000 7f24e79f3640 1 -- 172.21.15.19:0/998361447 wait complete. 2024-04-25T10:51:55.610 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:55 smithi169 ceph-mon[33049]: Deploying daemon mds.l on smithi169 2024-04-25T10:51:55.611 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:55 smithi169 ceph-mon[33049]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:55.611 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:55 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3994362191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:51:55.611 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:55 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/998361447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:51:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:55 smithi112 ceph-mon[32799]: Deploying daemon mds.l on smithi169 2024-04-25T10:51:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:55 smithi112 ceph-mon[32799]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:55 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3994362191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:51:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:55 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/998361447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:51:55.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:55 smithi019 ceph-mon[29209]: Deploying daemon mds.l on smithi169 2024-04-25T10:51:55.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:55 smithi019 ceph-mon[29209]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:55.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:55 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3994362191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:51:55.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:55 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/998361447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:51:56.001 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:55 smithi169 systemd[1]: Starting Ceph mds.l for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:51:56.260 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 podman[62957]: 2024-04-25 10:51:55.909901759 +0000 UTC m=+0.018023375 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:51:56.260 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 podman[62957]: 2024-04-25 10:51:56.021329298 +0000 UTC m=+0.129450918 container create 6ffd791b24c684ac580482b3ee04d0376db93c10ed1b0784618a9cc10de1fe07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T10:51:56.261 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 podman[62957]: 2024-04-25 10:51:56.257220477 +0000 UTC m=+0.365342093 container init 6ffd791b24c684ac580482b3ee04d0376db93c10ed1b0784618a9cc10de1fe07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:51:56.261 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 podman[62957]: 2024-04-25 10:51:56.259787703 +0000 UTC m=+0.367909320 container start 6ffd791b24c684ac580482b3ee04d0376db93c10ed1b0784618a9cc10de1fe07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T10:51:56.751 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 ceph-mds[62976]: -- 172.21.15.169:0/876323213 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55948818fa40 con 0x5594881d7600 2024-04-25T10:51:56.751 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l[62972]: starting mds.l at 2024-04-25T10:51:56.751 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 bash[62957]: 6ffd791b24c684ac580482b3ee04d0376db93c10ed1b0784618a9cc10de1fe07 2024-04-25T10:51:56.752 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 10:51:56 smithi169 systemd[1]: Started Ceph mds.l for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:51:56.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:51:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 -- 172.21.15.19:0/2275058354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104103b10 msgr2=0x7f5104104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2275058354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104103b10 0x7f5104104810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5100009900 tx=0x7f510002a410 comp rx=0 tx=0).stop 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 -- 172.21.15.19:0/2275058354 shutdown_connections 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2275058354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5104109a50 0x7f510410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2275058354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104104d50 0x7f5104105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2275058354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104103b10 0x7f5104104810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 -- 172.21.15.19:0/2275058354 >> 172.21.15.19:0/2275058354 conn(0x7f51040fed40 msgr2=0x7f51041011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 -- 172.21.15.19:0/2275058354 shutdown_connections 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.077+0000 7f510bf6c640 1 -- 172.21.15.19:0/2275058354 wait complete. 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 Processor -- start 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 -- start start 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104103b10 0x7f510419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104104d50 0x7f510419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5104109a50 0x7f51041a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f510410f610 con 0x7f5104104d50 2024-04-25T10:51:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f510410f490 con 0x7f5104103b10 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f510bf6c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f510410f790 con 0x7f5104109a50 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f5109507640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104104d50 0x7f510419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f5109507640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104104d50 0x7f510419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58372/0 (socket says 172.21.15.19:58372) 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.078+0000 7f5109507640 1 -- 172.21.15.19:0/2261386246 learned_addr learned my addr 172.21.15.19:0/2261386246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f5109d08640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104103b10 0x7f510419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f510a509640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5104109a50 0x7f51041a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f5109507640 1 -- 172.21.15.19:0/2261386246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5104109a50 msgr2=0x7f51041a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f5109507640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5104109a50 0x7f51041a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f5109507640 1 -- 172.21.15.19:0/2261386246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104103b10 msgr2=0x7f510419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f5109507640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104103b10 0x7f510419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f5109507640 1 -- 172.21.15.19:0/2261386246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51041a1d90 con 0x7f5104104d50 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f5109507640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104104d50 0x7f510419a790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50f400d950 tx=0x7f50f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f50faffd640 1 -- 172.21.15.19:0/2261386246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4017070 con 0x7f5104104d50 2024-04-25T10:51:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f50faffd640 1 -- 172.21.15.19:0/2261386246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f4009c30 con 0x7f5104104d50 2024-04-25T10:51:57.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51041a2080 con 0x7f5104104d50 2024-04-25T10:51:57.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f50faffd640 1 -- 172.21.15.19:0/2261386246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f400f620 con 0x7f5104104d50 2024-04-25T10:51:57.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.079+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51041a25d0 con 0x7f5104104d50 2024-04-25T10:51:57.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.080+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5104108860 con 0x7f5104104d50 2024-04-25T10:51:57.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.082+0000 7f50faffd640 1 -- 172.21.15.19:0/2261386246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f50f4010460 con 0x7f5104104d50 2024-04-25T10:51:57.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.083+0000 7f50faffd640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f50f00afaf0 0x7f50f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:57.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.083+0000 7f5109d08640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f50f00afaf0 0x7f50f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:57.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.083+0000 7f50faffd640 1 -- 172.21.15.19:0/2261386246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f50f40de350 con 0x7f5104104d50 2024-04-25T10:51:57.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.084+0000 7f5109d08640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f50f00afaf0 0x7f50f00b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5100005ab0 tx=0x7f5100031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.087+0000 7f50faffd640 1 -- 172.21.15.19:0/2261386246 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50f4009de0 con 0x7f5104104d50 2024-04-25T10:51:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.312+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f510402db50 con 0x7f5104104d50 2024-04-25T10:51:57.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.313+0000 7f50faffd640 1 -- 172.21.15.19:0/2261386246 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f50f40a6ce0 con 0x7f5104104d50 2024-04-25T10:51:57.314 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:51:57.314 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:51:57.314 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f50f00afaf0 msgr2=0x7f50f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f50f00afaf0 0x7f50f00b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5100005ab0 tx=0x7f5100031400 comp rx=0 tx=0).stop 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104104d50 msgr2=0x7f510419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104104d50 0x7f510419a790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50f400d950 tx=0x7f50f400de10 comp rx=0 tx=0).stop 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 shutdown_connections 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5104109a50 0x7f51041a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f50f00afaf0 0x7f50f00b2000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5104104d50 0x7f510419a790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 --2- 172.21.15.19:0/2261386246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5104103b10 0x7f510419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 >> 172.21.15.19:0/2261386246 conn(0x7f51040fed40 msgr2=0x7f5104105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:57.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 shutdown_connections 2024-04-25T10:51:57.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:57.315+0000 7f510bf6c640 1 -- 172.21.15.19:0/2261386246 wait complete. 2024-04-25T10:51:57.374 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:57.374 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.169:6832/894672787,v1:172.21.15.169:6833/894672787] up:boot 2024-04-25T10:51:57.374 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: fsmap 1 up:standby 2024-04-25T10:51:57.374 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:51:57.374 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.375 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.375 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.375 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:51:57.375 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:51:57.375 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:57.375 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:57 smithi112 ceph-mon[32799]: Deploying daemon mds.k on smithi112 2024-04-25T10:51:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.169:6832/894672787,v1:172.21.15.169:6833/894672787] up:boot 2024-04-25T10:51:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: fsmap 1 up:standby 2024-04-25T10:51:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:51:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:51:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:51:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:57.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:57 smithi019 ceph-mon[29209]: Deploying daemon mds.k on smithi112 2024-04-25T10:51:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.169:6832/894672787,v1:172.21.15.169:6833/894672787] up:boot 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: fsmap 1 up:standby 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:51:57.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:57 smithi169 ceph-mon[33049]: Deploying daemon mds.k on smithi112 2024-04-25T10:51:58.437 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:58 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2261386246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:51:58.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:58 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2261386246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:51:58.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:58 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2261386246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:51:58.809 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:58 smithi112 systemd[1]: Starting Ceph mds.k for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:51:59.141 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:58 smithi112 podman[61614]: 2024-04-25 10:51:58.730704091 +0000 UTC m=+0.017975451 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:51:59.141 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:58 smithi112 podman[61614]: 2024-04-25 10:51:58.938332515 +0000 UTC m=+0.225603910 container create 8683e53191b0153b7c93a37c5ed2d2425589bf7be08331c603011fe96142f9f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T10:51:59.141 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:59 smithi112 podman[61614]: 2024-04-25 10:51:59.140829793 +0000 UTC m=+0.428101146 container init 8683e53191b0153b7c93a37c5ed2d2425589bf7be08331c603011fe96142f9f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T10:51:59.328 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:51:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 -- 172.21.15.19:0/2191288946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d4103c00 msgr2=0x7f40d4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 --2- 172.21.15.19:0/2191288946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d4103c00 0x7f40d4104010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009900 tx=0x7f40d002a410 comp rx=0 tx=0).stop 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 -- 172.21.15.19:0/2191288946 shutdown_connections 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 --2- 172.21.15.19:0/2191288946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d4108900 0x7f40d410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 --2- 172.21.15.19:0/2191288946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d4103c00 0x7f40d4104010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 --2- 172.21.15.19:0/2191288946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d41029c0 0x7f40d41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 -- 172.21.15.19:0/2191288946 >> 172.21.15.19:0/2191288946 conn(0x7f40d40fdbd0 msgr2=0x7f40d4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 -- 172.21.15.19:0/2191288946 shutdown_connections 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.458+0000 7f40dc16c640 1 -- 172.21.15.19:0/2191288946 wait complete. 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 Processor -- start 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 -- start start 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d41029c0 0x7f40d419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d4103c00 0x7f40d419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d4108900 0x7f40d41a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40d410f5f0 con 0x7f40d41029c0 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40d410f470 con 0x7f40d4108900 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40dc16c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40d410f770 con 0x7f40d4103c00 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40d9f08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d41029c0 0x7f40d419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40d9f08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d41029c0 0x7f40d419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58394/0 (socket says 172.21.15.19:58394) 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.459+0000 7f40d9f08640 1 -- 172.21.15.19:0/3288039776 learned_addr learned my addr 172.21.15.19:0/3288039776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40d9707640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d4103c00 0x7f40d419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40da709640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d4108900 0x7f40d41a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40d9f08640 1 -- 172.21.15.19:0/3288039776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d4103c00 msgr2=0x7f40d419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40d9f08640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d4103c00 0x7f40d419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40d9f08640 1 -- 172.21.15.19:0/3288039776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d4108900 msgr2=0x7f40d41a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40d9f08640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d4108900 0x7f40d41a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40d9f08640 1 -- 172.21.15.19:0/3288039776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d41a1d50 con 0x7f40d41029c0 2024-04-25T10:51:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40d9f08640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d41029c0 0x7f40d419a210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e9f0 tx=0x7f40c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40c6ffd640 1 -- 172.21.15.19:0/3288039776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c800ccc0 con 0x7f40d41029c0 2024-04-25T10:51:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40c6ffd640 1 -- 172.21.15.19:0/3288039776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c80035f0 con 0x7f40d41029c0 2024-04-25T10:51:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d41a1fe0 con 0x7f40d41029c0 2024-04-25T10:51:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40c6ffd640 1 -- 172.21.15.19:0/3288039776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c800ce60 con 0x7f40d41029c0 2024-04-25T10:51:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.460+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d41a24a0 con 0x7f40d41029c0 2024-04-25T10:51:59.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.461+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d4107710 con 0x7f40d41029c0 2024-04-25T10:51:59.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.463+0000 7f40c6ffd640 1 -- 172.21.15.19:0/3288039776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f40c8003aa0 con 0x7f40d41029c0 2024-04-25T10:51:59.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.464+0000 7f40c6ffd640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c00afaf0 0x7f40c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:51:59.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.464+0000 7f40d9707640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c00afaf0 0x7f40c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:51:59.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.464+0000 7f40c6ffd640 1 -- 172.21.15.19:0/3288039776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f40c80deaf0 con 0x7f40d41029c0 2024-04-25T10:51:59.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.464+0000 7f40d9707640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c00afaf0 0x7f40c00b2000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009230 tx=0x7f40d0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:51:59.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.469+0000 7f40c6ffd640 1 -- 172.21.15.19:0/3288039776 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40c80a7b70 con 0x7f40d41029c0 2024-04-25T10:51:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:59 smithi112 ceph-mon[32799]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:59 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:51:59 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:59.559 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:59 smithi112 podman[61614]: 2024-04-25 10:51:59.143547856 +0000 UTC m=+0.430819214 container start 8683e53191b0153b7c93a37c5ed2d2425589bf7be08331c603011fe96142f9f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:51:59.559 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:59 smithi112 ceph-mds[61633]: -- 172.21.15.112:0/2089866712 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559669a09a40 con 0x559669a51600 2024-04-25T10:51:59.559 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k[61629]: starting mds.k at 2024-04-25T10:51:59.560 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:59 smithi112 bash[61614]: 8683e53191b0153b7c93a37c5ed2d2425589bf7be08331c603011fe96142f9f4 2024-04-25T10:51:59.560 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 10:51:59 smithi112 systemd[1]: Started Ceph mds.k for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:51:59.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:59 smithi019 ceph-mon[29209]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:59.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:59.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:51:59 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:59.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.696+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40d402db50 con 0x7f40d41029c0 2024-04-25T10:51:59.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.697+0000 7f40c6ffd640 1 -- 172.21.15.19:0/3288039776 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f40c80a7470 con 0x7f40d41029c0 2024-04-25T10:51:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:51:59.698 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:51:59.698 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-25T10:51:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c00afaf0 msgr2=0x7f40c00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c00afaf0 0x7f40c00b2000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009230 tx=0x7f40d0031400 comp rx=0 tx=0).stop 2024-04-25T10:51:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d41029c0 msgr2=0x7f40d419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:51:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d41029c0 0x7f40d419a210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e9f0 tx=0x7f40c800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 shutdown_connections 2024-04-25T10:51:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d4108900 0x7f40d41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d4103c00 0x7f40d419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c00afaf0 0x7f40c00b2000 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 --2- 172.21.15.19:0/3288039776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40d41029c0 0x7f40d419a210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:51:59.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 >> 172.21.15.19:0/3288039776 conn(0x7f40d40fdbd0 msgr2=0x7f40d41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:51:59.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 shutdown_connections 2024-04-25T10:51:59.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:51:59.699+0000 7f40dc16c640 1 -- 172.21.15.19:0/3288039776 wait complete. 2024-04-25T10:51:59.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:59 smithi169 ceph-mon[33049]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:51:59.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:59 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:51:59.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:51:59 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.112:6832/1523385162,v1:172.21.15.112:6833/1523385162] up:boot 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: fsmap 2 up:standby 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: Deploying daemon mds.j on smithi019 2024-04-25T10:52:00.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:00 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3288039776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.112:6832/1523385162,v1:172.21.15.112:6833/1523385162] up:boot 2024-04-25T10:52:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: fsmap 2 up:standby 2024-04-25T10:52:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:52:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:00.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:00.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: Deploying daemon mds.j on smithi019 2024-04-25T10:52:00.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:00 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3288039776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:00.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.112:6832/1523385162,v1:172.21.15.112:6833/1523385162] up:boot 2024-04-25T10:52:00.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: fsmap 2 up:standby 2024-04-25T10:52:00.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:52:00.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:00.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:00.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:00.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:00.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: Deploying daemon mds.j on smithi019 2024-04-25T10:52:00.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:00 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3288039776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:01.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:01 smithi019 ceph-mon[29209]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:01.612 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:01 smithi019 systemd[1]: Starting Ceph mds.j for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:01.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:01.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:01 smithi169 ceph-mon[33049]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:01.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:01 smithi112 ceph-mon[32799]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.827+0000 7f673b30d640 1 -- 172.21.15.19:0/3005219325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734109790 msgr2=0x7f673410bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.827+0000 7f673b30d640 1 --2- 172.21.15.19:0/3005219325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734109790 0x7f673410bca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f672c009ca0 tx=0x7f672c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 -- 172.21.15.19:0/3005219325 shutdown_connections 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 --2- 172.21.15.19:0/3005219325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734109790 0x7f673410bca0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 --2- 172.21.15.19:0/3005219325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6734106c20 0x7f6734109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 --2- 172.21.15.19:0/3005219325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734101e20 0x7f6734102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 -- 172.21.15.19:0/3005219325 >> 172.21.15.19:0/3005219325 conn(0x7f67340fd3b0 msgr2=0x7f67340ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 -- 172.21.15.19:0/3005219325 shutdown_connections 2024-04-25T10:52:01.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 -- 172.21.15.19:0/3005219325 wait complete. 2024-04-25T10:52:01.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 Processor -- start 2024-04-25T10:52:01.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.828+0000 7f673b30d640 1 -- start start 2024-04-25T10:52:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f673b30d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734101e20 0x7f673419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f673b30d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734106c20 0x7f673419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f673b30d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6734109790 0x7f67341a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f673b30d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f673410f600 con 0x7f6734106c20 2024-04-25T10:52:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f673b30d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f673410f480 con 0x7f6734109790 2024-04-25T10:52:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f673b30d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f673410f780 con 0x7f6734101e20 2024-04-25T10:52:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734106c20 0x7f673419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734106c20 0x7f673419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58410/0 (socket says 172.21.15.19:58410) 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 -- 172.21.15.19:0/2194098503 learned_addr learned my addr 172.21.15.19:0/2194098503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67398aa640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6734109790 0x7f67341a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67390a9640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734101e20 0x7f673419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 -- 172.21.15.19:0/2194098503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734101e20 msgr2=0x7f673419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734101e20 0x7f673419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 -- 172.21.15.19:0/2194098503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6734109790 msgr2=0x7f67341a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6734109790 0x7f67341a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 -- 172.21.15.19:0/2194098503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67341a1c90 con 0x7f6734106c20 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67390a9640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734101e20 0x7f673419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.829+0000 7f67388a8640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734106c20 0x7f673419a690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f672400e9f0 tx=0x7f672400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.830+0000 7f672a7fc640 1 -- 172.21.15.19:0/2194098503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400cd80 con 0x7f6734106c20 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.830+0000 7f672a7fc640 1 -- 172.21.15.19:0/2194098503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67240108e0 con 0x7f6734106c20 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.830+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67341a1f80 con 0x7f6734106c20 2024-04-25T10:52:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.830+0000 7f672a7fc640 1 -- 172.21.15.19:0/2194098503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724018980 con 0x7f6734106c20 2024-04-25T10:52:01.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.830+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67341a23f0 con 0x7f6734106c20 2024-04-25T10:52:01.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.831+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67340feed0 con 0x7f6734106c20 2024-04-25T10:52:01.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.833+0000 7f672a7fc640 1 -- 172.21.15.19:0/2194098503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6724010430 con 0x7f6734106c20 2024-04-25T10:52:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.834+0000 7f672a7fc640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67200afaf0 0x7f67200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.834+0000 7f67390a9640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67200afaf0 0x7f67200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.834+0000 7f672a7fc640 1 -- 172.21.15.19:0/2194098503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f67240defe0 con 0x7f6734106c20 2024-04-25T10:52:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.834+0000 7f67390a9640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67200afaf0 0x7f67200b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67300058c0 tx=0x7f6730005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:01.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:01.837+0000 7f672a7fc640 1 -- 172.21.15.19:0/2194098503 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67240a8060 con 0x7f6734106c20 2024-04-25T10:52:01.977 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:01 smithi019 podman[69928]: 2024-04-25 10:52:01.624539961 +0000 UTC m=+0.018935218 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:01.977 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:01 smithi019 podman[69928]: 2024-04-25 10:52:01.724414484 +0000 UTC m=+0.118809723 container create 27ae0acb694b40d180d33f0730f3a83afdbd2ee74cdcecaf42cfcdc3eb500bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T10:52:01.977 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:01 smithi019 podman[69928]: 2024-04-25 10:52:01.977207804 +0000 UTC m=+0.371603042 container init 27ae0acb694b40d180d33f0730f3a83afdbd2ee74cdcecaf42cfcdc3eb500bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T10:52:02.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.065+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f673402db50 con 0x7f6734106c20 2024-04-25T10:52:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.066+0000 7f672a7fc640 1 -- 172.21.15.19:0/2194098503 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f67240b00e0 con 0x7f6734106c20 2024-04-25T10:52:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:02.067 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:52:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-25T10:52:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67200afaf0 msgr2=0x7f67200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67200afaf0 0x7f67200b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67300058c0 tx=0x7f6730005850 comp rx=0 tx=0).stop 2024-04-25T10:52:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734106c20 msgr2=0x7f673419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734106c20 0x7f673419a690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f672400e9f0 tx=0x7f672400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:52:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 shutdown_connections 2024-04-25T10:52:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6734109790 0x7f67341a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67200afaf0 0x7f67200b2000 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6734106c20 0x7f673419a690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 --2- 172.21.15.19:0/2194098503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734101e20 0x7f673419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 >> 172.21.15.19:0/2194098503 conn(0x7f67340fd3b0 msgr2=0x7f6734107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 shutdown_connections 2024-04-25T10:52:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:02.068+0000 7f673b30d640 1 -- 172.21.15.19:0/2194098503 wait complete. 2024-04-25T10:52:02.337 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:01 smithi019 podman[69928]: 2024-04-25 10:52:01.980064901 +0000 UTC m=+0.374460145 container start 27ae0acb694b40d180d33f0730f3a83afdbd2ee74cdcecaf42cfcdc3eb500bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T10:52:02.337 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mds[69986]: -- 172.21.15.19:0/466933102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fb32ada40 con 0x563fb32f5a80 2024-04-25T10:52:02.337 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j[69982]: starting mds.j at 2024-04-25T10:52:02.337 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:02 smithi019 bash[69928]: 27ae0acb694b40d180d33f0730f3a83afdbd2ee74cdcecaf42cfcdc3eb500bb4 2024-04-25T10:52:02.337 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 10:52:02 smithi019 systemd[1]: Started Ceph mds.j for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2194098503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:02.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:02.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:02 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:02.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:02 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2194098503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:02.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:02.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:02.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:02 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:02.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:02 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2194098503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:02.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:02.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:02.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:02.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:02 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:03 smithi019 ceph-mon[29209]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:03 smithi019 ceph-mon[29209]: Deploying daemon mds.d on smithi019 2024-04-25T10:52:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:03 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.19:6834/311381787,v1:172.21.15.19:6835/311381787] up:boot 2024-04-25T10:52:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:03 smithi019 ceph-mon[29209]: fsmap 3 up:standby 2024-04-25T10:52:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:03 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:52:03.738 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:03 smithi169 ceph-mon[33049]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:03.738 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:03 smithi169 ceph-mon[33049]: Deploying daemon mds.d on smithi019 2024-04-25T10:52:03.738 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:03 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.19:6834/311381787,v1:172.21.15.19:6835/311381787] up:boot 2024-04-25T10:52:03.738 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:03 smithi169 ceph-mon[33049]: fsmap 3 up:standby 2024-04-25T10:52:03.738 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:03 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:52:03.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:03 smithi112 ceph-mon[32799]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:03.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:03 smithi112 ceph-mon[32799]: Deploying daemon mds.d on smithi019 2024-04-25T10:52:03.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:03 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.19:6834/311381787,v1:172.21.15.19:6835/311381787] up:boot 2024-04-25T10:52:03.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:03 smithi112 ceph-mon[32799]: fsmap 3 up:standby 2024-04-25T10:52:03.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:03 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:52:04.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:04.361 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 systemd[1]: Starting Ceph mds.d for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 -- 172.21.15.19:0/1267144319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 msgr2=0x7f1a6810bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 --2- 172.21.15.19:0/1267144319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a6810bc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009ca0 tx=0x7f1a6002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 -- 172.21.15.19:0/1267144319 shutdown_connections 2024-04-25T10:52:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 --2- 172.21.15.19:0/1267144319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a6810bc60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 --2- 172.21.15.19:0/1267144319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a68106be0 0x7f1a681090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 --2- 172.21.15.19:0/1267144319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a681041f0 0x7f1a68106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 -- 172.21.15.19:0/1267144319 >> 172.21.15.19:0/1267144319 conn(0x7f1a680fdbd0 msgr2=0x7f1a68100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 -- 172.21.15.19:0/1267144319 shutdown_connections 2024-04-25T10:52:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.363+0000 7f1a70405640 1 -- 172.21.15.19:0/1267144319 wait complete. 2024-04-25T10:52:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.364+0000 7f1a70405640 1 Processor -- start 2024-04-25T10:52:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.364+0000 7f1a70405640 1 -- start start 2024-04-25T10:52:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.364+0000 7f1a70405640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a681041f0 0x7f1a6819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.364+0000 7f1a70405640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a68106be0 0x7f1a6819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a70405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a681a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a70405640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6810f5c0 con 0x7f1a68109750 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a70405640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6810f440 con 0x7f1a68106be0 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a70405640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6810f740 con 0x7f1a681041f0 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a681a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a681a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42746/0 (socket says 172.21.15.19:42746) 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 -- 172.21.15.19:0/1514566998 learned_addr learned my addr 172.21.15.19:0/1514566998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e1a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a681041f0 0x7f1a6819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6d9a0640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a68106be0 0x7f1a6819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 -- 172.21.15.19:0/1514566998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a681041f0 msgr2=0x7f1a6819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a681041f0 0x7f1a6819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 -- 172.21.15.19:0/1514566998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a68106be0 msgr2=0x7f1a6819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a68106be0 0x7f1a6819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 -- 172.21.15.19:0/1514566998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a681a1bf0 con 0x7f1a68109750 2024-04-25T10:52:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.365+0000 7f1a6e9a2640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a681a1630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a600097b0 tx=0x7f1a60003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.366+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/1514566998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60004240 con 0x7f1a68109750 2024-04-25T10:52:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.366+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/1514566998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6002f380 con 0x7f1a68109750 2024-04-25T10:52:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.366+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a681a1e80 con 0x7f1a68109750 2024-04-25T10:52:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.366+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/1514566998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6002fc20 con 0x7f1a68109750 2024-04-25T10:52:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.366+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a681a2340 con 0x7f1a68109750 2024-04-25T10:52:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.367+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a680ff6f0 con 0x7f1a68109750 2024-04-25T10:52:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.369+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/1514566998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1a600043e0 con 0x7f1a68109750 2024-04-25T10:52:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.369+0000 7f1a5f7fe640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a540afb70 0x7f1a540b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.370+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/1514566998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f1a60100170 con 0x7f1a68109750 2024-04-25T10:52:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.370+0000 7f1a6e1a1640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a540afb70 0x7f1a540b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.370+0000 7f1a6e1a1640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a540afb70 0x7f1a540b2080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005fd0 tx=0x7f1a6400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.373+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/1514566998 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a600c91f0 con 0x7f1a68109750 2024-04-25T10:52:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.610+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a6802db50 con 0x7f1a68109750 2024-04-25T10:52:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.611+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/1514566998 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f1a600c8af0 con 0x7f1a68109750 2024-04-25T10:52:04.612 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:52:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-04-25T10:52:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a540afb70 msgr2=0x7f1a540b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a540afb70 0x7f1a540b2080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005fd0 tx=0x7f1a6400b390 comp rx=0 tx=0).stop 2024-04-25T10:52:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 msgr2=0x7f1a681a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a681a1630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a600097b0 tx=0x7f1a60003c20 comp rx=0 tx=0).stop 2024-04-25T10:52:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 shutdown_connections 2024-04-25T10:52:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a540afb70 0x7f1a540b2080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a68109750 0x7f1a681a1630 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a68106be0 0x7f1a6819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 --2- 172.21.15.19:0/1514566998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a681041f0 0x7f1a6819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 >> 172.21.15.19:0/1514566998 conn(0x7f1a680fdbd0 msgr2=0x7f1a68104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 shutdown_connections 2024-04-25T10:52:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:04.613+0000 7f1a70405640 1 -- 172.21.15.19:0/1514566998 wait complete. 2024-04-25T10:52:04.823 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 podman[70318]: 2024-04-25 10:52:04.418500978 +0000 UTC m=+0.018643415 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:04.824 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 podman[70318]: 2024-04-25 10:52:04.534477375 +0000 UTC m=+0.134619803 container create 9fa8ac13a722282daa64aa2898623220d9577955e3d6ff7429dda1d943495037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T10:52:04.824 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 podman[70318]: 2024-04-25 10:52:04.723311295 +0000 UTC m=+0.323453723 container init 9fa8ac13a722282daa64aa2898623220d9577955e3d6ff7429dda1d943495037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T10:52:04.824 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 podman[70318]: 2024-04-25 10:52:04.726120925 +0000 UTC m=+0.326263353 container start 9fa8ac13a722282daa64aa2898623220d9577955e3d6ff7429dda1d943495037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:52:04.824 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 ceph-mds[70339]: -- 172.21.15.19:0/469547327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8fbe0fa40 con 0x55e8fbe57180 2024-04-25T10:52:04.824 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d[70335]: starting mds.d at 2024-04-25T10:52:04.824 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 bash[70318]: 9fa8ac13a722282daa64aa2898623220d9577955e3d6ff7429dda1d943495037 2024-04-25T10:52:05.111 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 10:52:04 smithi019 systemd[1]: Started Ceph mds.d for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:05.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:05.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1514566998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:05.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:05.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:05.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:05 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:05.694 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:05.694 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1514566998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:05.694 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.694 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.695 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.695 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:05.695 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:05.695 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:05 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1514566998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:05.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:05.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:05.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:05 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:06 smithi112 ceph-mon[32799]: Deploying daemon mds.e on smithi112 2024-04-25T10:52:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:06 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.19:6836/3384911547,v1:172.21.15.19:6837/3384911547] up:boot 2024-04-25T10:52:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:06 smithi112 ceph-mon[32799]: fsmap 4 up:standby 2024-04-25T10:52:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:06 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:52:06.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:06 smithi019 ceph-mon[29209]: Deploying daemon mds.e on smithi112 2024-04-25T10:52:06.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:06 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.19:6836/3384911547,v1:172.21.15.19:6837/3384911547] up:boot 2024-04-25T10:52:06.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:06 smithi019 ceph-mon[29209]: fsmap 4 up:standby 2024-04-25T10:52:06.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:06 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:52:06.626 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:06.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:06 smithi169 ceph-mon[33049]: Deploying daemon mds.e on smithi112 2024-04-25T10:52:06.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:06 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.19:6836/3384911547,v1:172.21.15.19:6837/3384911547] up:boot 2024-04-25T10:52:06.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:06 smithi169 ceph-mon[33049]: fsmap 4 up:standby 2024-04-25T10:52:06.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:06 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:52:06.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.755+0000 7f8d98e5a640 1 -- 172.21.15.19:0/3235809556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d94108900 msgr2=0x7f8d9410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.755+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/3235809556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d94108900 0x7f8d9410ad20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009900 tx=0x7f8d8c02a410 comp rx=0 tx=0).stop 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.755+0000 7f8d98e5a640 1 -- 172.21.15.19:0/3235809556 shutdown_connections 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.755+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/3235809556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d94108900 0x7f8d9410ad20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.755+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/3235809556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94103c00 0x7f8d94104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.755+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/3235809556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d941029c0 0x7f8d941036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.755+0000 7f8d98e5a640 1 -- 172.21.15.19:0/3235809556 >> 172.21.15.19:0/3235809556 conn(0x7f8d940fdbd0 msgr2=0x7f8d94100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.756+0000 7f8d98e5a640 1 -- 172.21.15.19:0/3235809556 shutdown_connections 2024-04-25T10:52:06.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.756+0000 7f8d98e5a640 1 -- 172.21.15.19:0/3235809556 wait complete. 2024-04-25T10:52:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.756+0000 7f8d98e5a640 1 Processor -- start 2024-04-25T10:52:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.756+0000 7f8d98e5a640 1 -- start start 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d98e5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d941029c0 0x7f8d9406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d98e5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94103c00 0x7f8d9419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d98e5a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d94108900 0x7f8d941a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d98e5a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9410e4d0 con 0x7f8d941029c0 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d98e5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9410e350 con 0x7f8d94103c00 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d98e5a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9410e650 con 0x7f8d94108900 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d941029c0 0x7f8d9406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d941029c0 0x7f8d9406a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42776/0 (socket says 172.21.15.19:42776) 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 -- 172.21.15.19:0/1037828497 learned_addr learned my addr 172.21.15.19:0/1037828497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d91d9b640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94103c00 0x7f8d9419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 -- 172.21.15.19:0/1037828497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d94108900 msgr2=0x7f8d941a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d94108900 0x7f8d941a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 -- 172.21.15.19:0/1037828497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94103c00 msgr2=0x7f8d9419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94103c00 0x7f8d9419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 -- 172.21.15.19:0/1037828497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d941a1c30 con 0x7f8d941029c0 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d9259c640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d941029c0 0x7f8d9406a900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400cce0 tx=0x7f8d8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d8b7fe640 1 -- 172.21.15.19:0/1037828497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8400f450 con 0x7f8d941029c0 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d8b7fe640 1 -- 172.21.15.19:0/1037828497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d8400faa0 con 0x7f8d941029c0 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.757+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d941a1f20 con 0x7f8d941029c0 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.758+0000 7f8d8b7fe640 1 -- 172.21.15.19:0/1037828497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84018b70 con 0x7f8d941029c0 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.758+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d941a23e0 con 0x7f8d941029c0 2024-04-25T10:52:06.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.759+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d94107710 con 0x7f8d941029c0 2024-04-25T10:52:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.760+0000 7f8d8b7fe640 1 -- 172.21.15.19:0/1037828497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8d84018d10 con 0x7f8d941029c0 2024-04-25T10:52:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.761+0000 7f8d8b7fe640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8d800afb70 0x7f8d800b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.761+0000 7f8d91d9b640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8d800afb70 0x7f8d800b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.761+0000 7f8d8b7fe640 1 -- 172.21.15.19:0/1037828497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8d840dec10 con 0x7f8d941029c0 2024-04-25T10:52:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.761+0000 7f8d91d9b640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8d800afb70 0x7f8d800b2080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d9406a2a0 tx=0x7f8d7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:06.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.764+0000 7f8d8b7fe640 1 -- 172.21.15.19:0/1037828497 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d84052ff0 con 0x7f8d941029c0 2024-04-25T10:52:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.997+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d9402de00 con 0x7f8d941029c0 2024-04-25T10:52:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:06.998+0000 7f8d8b7fe640 1 -- 172.21.15.19:0/1037828497 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f8d84014030 con 0x7f8d941029c0 2024-04-25T10:52:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:06.999 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:52:06.999 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8d800afb70 msgr2=0x7f8d800b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8d800afb70 0x7f8d800b2080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d9406a2a0 tx=0x7f8d7c005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d941029c0 msgr2=0x7f8d9406a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d941029c0 0x7f8d9406a900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400cce0 tx=0x7f8d8400a620 comp rx=0 tx=0).stop 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 shutdown_connections 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d94108900 0x7f8d941a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94103c00 0x7f8d9419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8d800afb70 0x7f8d800b2080 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 --2- 172.21.15.19:0/1037828497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d941029c0 0x7f8d9406a900 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 >> 172.21.15.19:0/1037828497 conn(0x7f8d940fdbd0 msgr2=0x7f8d941048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:07.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 shutdown_connections 2024-04-25T10:52:07.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:07.000+0000 7f8d98e5a640 1 -- 172.21.15.19:0/1037828497 wait complete. 2024-04-25T10:52:07.253 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 systemd[1]: Starting Ceph mds.e for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:07.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:07 smithi112 ceph-mon[32799]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:07.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:07 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1037828497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:07.522 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 podman[61935]: 2024-04-25 10:52:07.155902712 +0000 UTC m=+0.018311217 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:07.523 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 podman[61935]: 2024-04-25 10:52:07.275332468 +0000 UTC m=+0.137741016 container create 3617a79f9c35483c250f3fbb90ce612a44a216850f6e298bed63c917f89aa5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T10:52:07.523 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 podman[61935]: 2024-04-25 10:52:07.485814911 +0000 UTC m=+0.348223417 container init 3617a79f9c35483c250f3fbb90ce612a44a216850f6e298bed63c917f89aa5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:52:07.523 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 podman[61935]: 2024-04-25 10:52:07.489124086 +0000 UTC m=+0.351532599 container start 3617a79f9c35483c250f3fbb90ce612a44a216850f6e298bed63c917f89aa5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T10:52:07.523 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 ceph-mds[61955]: -- 172.21.15.112:0/2443900469 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601560bda40 con 0x560156105600 2024-04-25T10:52:07.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:07 smithi019 ceph-mon[29209]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:07.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:07 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1037828497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:07 smithi169 ceph-mon[33049]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:07 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1037828497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:07.809 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e[61951]: starting mds.e at 2024-04-25T10:52:07.809 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 bash[61935]: 3617a79f9c35483c250f3fbb90ce612a44a216850f6e298bed63c917f89aa5b0 2024-04-25T10:52:07.809 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 10:52:07 smithi112 systemd[1]: Started Ceph mds.e for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:08.988 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:08.988 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:08.988 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:08.988 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:08.988 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:08.989 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:08.989 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: Deploying daemon mds.f on smithi169 2024-04-25T10:52:08.989 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.112:6834/392627197,v1:172.21.15.112:6835/392627197] up:boot 2024-04-25T10:52:08.989 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: fsmap 5 up:standby 2024-04-25T10:52:08.989 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:08 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:52:09.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: Deploying daemon mds.f on smithi169 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.112:6834/392627197,v1:172.21.15.112:6835/392627197] up:boot 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: fsmap 5 up:standby 2024-04-25T10:52:09.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:08 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: Deploying daemon mds.f on smithi169 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.112:6834/392627197,v1:172.21.15.112:6835/392627197] up:boot 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: fsmap 5 up:standby 2024-04-25T10:52:09.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:08 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:52:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 -- 172.21.15.19:0/1843459769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c108920 msgr2=0x7ffb5c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/1843459769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c108920 0x7ffb5c10ad40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009ca0 tx=0x7ffb5402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 -- 172.21.15.19:0/1843459769 shutdown_connections 2024-04-25T10:52:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/1843459769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c108920 0x7ffb5c10ad40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/1843459769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c103c20 0x7ffb5c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/1843459769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb5c1029e0 0x7ffb5c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 -- 172.21.15.19:0/1843459769 >> 172.21.15.19:0/1843459769 conn(0x7ffb5c0fdbd0 msgr2=0x7ffb5c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 -- 172.21.15.19:0/1843459769 shutdown_connections 2024-04-25T10:52:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.141+0000 7ffb60e6e640 1 -- 172.21.15.19:0/1843459769 wait complete. 2024-04-25T10:52:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 Processor -- start 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 -- start start 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb5c1029e0 0x7ffb5c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c103c20 0x7ffb5c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c108920 0x7ffb5c1a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c10e4f0 con 0x7ffb5c103c20 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c10e370 con 0x7ffb5c108920 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb60e6e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c10e670 con 0x7ffb5c1029e0 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.142+0000 7ffb59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c103c20 0x7ffb5c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c103c20 0x7ffb5c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42800/0 (socket says 172.21.15.19:42800) 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 -- 172.21.15.19:0/949210454 learned_addr learned my addr 172.21.15.19:0/949210454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb5ad9d640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c108920 0x7ffb5c1a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb5a59c640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb5c1029e0 0x7ffb5c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 -- 172.21.15.19:0/949210454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb5c1029e0 msgr2=0x7ffb5c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb5c1029e0 0x7ffb5c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 -- 172.21.15.19:0/949210454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c108920 msgr2=0x7ffb5c1a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c108920 0x7ffb5c1a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 -- 172.21.15.19:0/949210454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c1a1d60 con 0x7ffb5c103c20 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb59d9b640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c103c20 0x7ffb5c19eff0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400d950 tx=0x7ffb4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb537fe640 1 -- 172.21.15.19:0/949210454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44017070 con 0x7ffb5c103c20 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb537fe640 1 -- 172.21.15.19:0/949210454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb44009c30 con 0x7ffb5c103c20 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb537fe640 1 -- 172.21.15.19:0/949210454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400f620 con 0x7ffb5c103c20 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5c1a1ff0 con 0x7ffb5c103c20 2024-04-25T10:52:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.143+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5c1a2530 con 0x7ffb5c103c20 2024-04-25T10:52:09.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.145+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb5c107730 con 0x7ffb5c103c20 2024-04-25T10:52:09.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.146+0000 7ffb537fe640 1 -- 172.21.15.19:0/949210454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ffb44010460 con 0x7ffb5c103c20 2024-04-25T10:52:09.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.147+0000 7ffb537fe640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ffb480afb70 0x7ffb480b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:09.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.147+0000 7ffb5a59c640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ffb480afb70 0x7ffb480b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:09.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.147+0000 7ffb537fe640 1 -- 172.21.15.19:0/949210454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ffb440de1d0 con 0x7ffb5c103c20 2024-04-25T10:52:09.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.147+0000 7ffb5a59c640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ffb480afb70 0x7ffb480b2080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005fd0 tx=0x7ffb4c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.150+0000 7ffb537fe640 1 -- 172.21.15.19:0/949210454 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb44009de0 con 0x7ffb5c103c20 2024-04-25T10:52:09.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.380+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb5c02db50 con 0x7ffb5c103c20 2024-04-25T10:52:09.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.381+0000 7ffb537fe640 1 -- 172.21.15.19:0/949210454 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7ffb440a6c80 con 0x7ffb5c103c20 2024-04-25T10:52:09.382 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:09.382 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T10:52:09.382 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-04-25T10:52:09.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ffb480afb70 msgr2=0x7ffb480b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:09.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ffb480afb70 0x7ffb480b2080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005fd0 tx=0x7ffb4c00b410 comp rx=0 tx=0).stop 2024-04-25T10:52:09.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c103c20 msgr2=0x7ffb5c19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:09.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c103c20 0x7ffb5c19eff0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400d950 tx=0x7ffb4400de10 comp rx=0 tx=0).stop 2024-04-25T10:52:09.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 shutdown_connections 2024-04-25T10:52:09.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c108920 0x7ffb5c1a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ffb480afb70 0x7ffb480b2080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5c103c20 0x7ffb5c19eff0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 --2- 172.21.15.19:0/949210454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb5c1029e0 0x7ffb5c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:09.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 >> 172.21.15.19:0/949210454 conn(0x7ffb5c0fdbd0 msgr2=0x7ffb5c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:09.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 shutdown_connections 2024-04-25T10:52:09.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:09.383+0000 7ffb60e6e640 1 -- 172.21.15.19:0/949210454 wait complete. 2024-04-25T10:52:09.991 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:09 smithi169 ceph-mon[33049]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:09.991 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:09 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/949210454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:09.992 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:09 smithi169 systemd[1]: Starting Ceph mds.f for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:10.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:09 smithi112 ceph-mon[32799]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:10.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:09 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/949210454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:10.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:09 smithi019 ceph-mon[29209]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:10.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:09 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/949210454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:10.251 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:09 smithi169 podman[63267]: 2024-04-25 10:52:09.990914015 +0000 UTC m=+0.107280736 container create 716ab5d21bb99c7589f2d07d0a39ef171a4b6a5dbc21b83170f78c77471bbdc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:52:10.251 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:09 smithi169 podman[63267]: 2024-04-25 10:52:09.90218045 +0000 UTC m=+0.018547151 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:10.252 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:10 smithi169 podman[63267]: 2024-04-25 10:52:10.234881267 +0000 UTC m=+0.351247945 container init 716ab5d21bb99c7589f2d07d0a39ef171a4b6a5dbc21b83170f78c77471bbdc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T10:52:10.252 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:10 smithi169 podman[63267]: 2024-04-25 10:52:10.237475616 +0000 UTC m=+0.353842306 container start 716ab5d21bb99c7589f2d07d0a39ef171a4b6a5dbc21b83170f78c77471bbdc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T10:52:10.751 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:10 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f[63282]: starting mds.f at 2024-04-25T10:52:10.751 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:10 smithi169 bash[63267]: 716ab5d21bb99c7589f2d07d0a39ef171a4b6a5dbc21b83170f78c77471bbdc0 2024-04-25T10:52:10.751 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 10:52:10 smithi169 systemd[1]: Started Ceph mds.f for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:11.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/2611281121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 msgr2=0x7f4c581036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/2611281121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581036e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c02a410 comp rx=0 tx=0).stop 2024-04-25T10:52:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/2611281121 shutdown_connections 2024-04-25T10:52:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/2611281121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c58108920 0x7f4c5810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/2611281121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c58103c20 0x7f4c58104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/2611281121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581036e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/2611281121 >> 172.21.15.19:0/2611281121 conn(0x7f4c580fdbf0 msgr2=0x7f4c58100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/2611281121 shutdown_connections 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.520+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/2611281121 wait complete. 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 Processor -- start 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 -- start start 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c58103c20 0x7f4c581996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c58108920 0x7f4c581a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5810e4e0 con 0x7f4c581029e0 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5810e360 con 0x7f4c58103c20 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5ef4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5810e660 con 0x7f4c58108920 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5cce9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5cce9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42820/0 (socket says 172.21.15.19:42820) 2024-04-25T10:52:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5cce9640 1 -- 172.21.15.19:0/995045149 learned_addr learned my addr 172.21.15.19:0/995045149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c57fff640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c58103c20 0x7f4c581996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.521+0000 7f4c5d4ea640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c58108920 0x7f4c581a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5cce9640 1 -- 172.21.15.19:0/995045149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c58108920 msgr2=0x7f4c581a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5cce9640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c58108920 0x7f4c581a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5cce9640 1 -- 172.21.15.19:0/995045149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c58103c20 msgr2=0x7f4c581996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5cce9640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c58103c20 0x7f4c581996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5cce9640 1 -- 172.21.15.19:0/995045149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c581a0ce0 con 0x7f4c581029e0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5cce9640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581991a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006010 tx=0x7f4c4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c55ffb640 1 -- 172.21.15.19:0/995045149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c03f030 con 0x7f4c581029e0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c581a0f70 con 0x7f4c581029e0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c55ffb640 1 -- 172.21.15.19:0/995045149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c003aa0 con 0x7f4c581029e0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c55ffb640 1 -- 172.21.15.19:0/995045149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c035070 con 0x7f4c581029e0 2024-04-25T10:52:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.522+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c58074fb0 con 0x7f4c581029e0 2024-04-25T10:52:11.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.523+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c580759d0 con 0x7f4c581029e0 2024-04-25T10:52:11.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.525+0000 7f4c55ffb640 1 -- 172.21.15.19:0/995045149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4c4c042030 con 0x7f4c581029e0 2024-04-25T10:52:11.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.525+0000 7f4c55ffb640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c480b1d00 0x7f4c480b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:11.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.526+0000 7f4c57fff640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c480b1d00 0x7f4c480b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:11.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.526+0000 7f4c55ffb640 1 -- 172.21.15.19:0/995045149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f4c4c0ff150 con 0x7f4c581029e0 2024-04-25T10:52:11.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.526+0000 7f4c57fff640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c480b1d00 0x7f4c480b4210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c5819a240 tx=0x7f4c4400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:11.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.529+0000 7f4c55ffb640 1 -- 172.21.15.19:0/995045149 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c4c0c7ae0 con 0x7f4c581029e0 2024-04-25T10:52:11.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:11.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: Deploying daemon mds.c on smithi169 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.169:6834/1662431671,v1:172.21.15.169:6835/1662431671] up:boot 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: fsmap 6 up:standby 2024-04-25T10:52:11.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:11 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:52:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.755+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c58075c20 con 0x7f4c581029e0 2024-04-25T10:52:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.756+0000 7f4c55ffb640 1 -- 172.21.15.19:0/995045149 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f4c4c0d01b0 con 0x7f4c581029e0 2024-04-25T10:52:11.756 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T10:52:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c480b1d00 msgr2=0x7f4c480b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c480b1d00 0x7f4c480b4210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c5819a240 tx=0x7f4c4400b410 comp rx=0 tx=0).stop 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 msgr2=0x7f4c581991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581991a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006010 tx=0x7f4c4c004270 comp rx=0 tx=0).stop 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 shutdown_connections 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c58108920 0x7f4c581a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c58103c20 0x7f4c581996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c480b1d00 0x7f4c480b4210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 --2- 172.21.15.19:0/995045149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c581029e0 0x7f4c581991a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 >> 172.21.15.19:0/995045149 conn(0x7f4c580fdbf0 msgr2=0x7f4c58104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 shutdown_connections 2024-04-25T10:52:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:11.758+0000 7f4c5ef4d640 1 -- 172.21.15.19:0/995045149 wait complete. 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:11.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: Deploying daemon mds.c on smithi169 2024-04-25T10:52:11.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.169:6834/1662431671,v1:172.21.15.169:6835/1662431671] up:boot 2024-04-25T10:52:11.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: fsmap 6 up:standby 2024-04-25T10:52:11.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:11 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: Deploying daemon mds.c on smithi169 2024-04-25T10:52:11.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.169:6834/1662431671,v1:172.21.15.169:6835/1662431671] up:boot 2024-04-25T10:52:11.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: fsmap 6 up:standby 2024-04-25T10:52:11.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:11 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:52:12.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:12 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/995045149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:12.743 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:12 smithi169 systemd[1]: Starting Ceph mds.c for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:12.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:12 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/995045149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:12.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:12 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/995045149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:13.002 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:12 smithi169 podman[63587]: 2024-04-25 10:52:12.644662704 +0000 UTC m=+0.018159979 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:13.002 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:12 smithi169 podman[63587]: 2024-04-25 10:52:12.744518244 +0000 UTC m=+0.118015548 container create 14a5ea9b29375a2d508cbf2528bf18dfc88391b00945ba23dbd61aaa1bbac761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T10:52:13.002 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:12 smithi169 podman[63587]: 2024-04-25 10:52:12.980360395 +0000 UTC m=+0.353857661 container init 14a5ea9b29375a2d508cbf2528bf18dfc88391b00945ba23dbd61aaa1bbac761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:52:13.002 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:12 smithi169 podman[63587]: 2024-04-25 10:52:12.9829677 +0000 UTC m=+0.356464964 container start 14a5ea9b29375a2d508cbf2528bf18dfc88391b00945ba23dbd61aaa1bbac761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T10:52:13.408 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mds[63606]: -- 172.21.15.169:0/3584426180 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631cf6dba40 con 0x5631cf723a80 2024-04-25T10:52:13.408 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c[63602]: starting mds.c at 2024-04-25T10:52:13.408 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:13 smithi169 bash[63587]: 14a5ea9b29375a2d508cbf2528bf18dfc88391b00945ba23dbd61aaa1bbac761 2024-04-25T10:52:13.408 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 10:52:13 smithi169 systemd[1]: Started Ceph mds.c for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mon[33049]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:13 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:13.771 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:13 smithi019 ceph-mon[29209]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:13 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:13.807 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:13 smithi112 ceph-mon[32799]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:13.807 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.807 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.807 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:13.807 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:13.807 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:13.807 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:13 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 -- 172.21.15.19:0/187566401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c104d40 msgr2=0x7fe18c105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 --2- 172.21.15.19:0/187566401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c104d40 0x7fe18c105150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe180009900 tx=0x7fe18002a410 comp rx=0 tx=0).stop 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 -- 172.21.15.19:0/187566401 shutdown_connections 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 --2- 172.21.15.19:0/187566401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18c109a40 0x7fe18c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 --2- 172.21.15.19:0/187566401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c104d40 0x7fe18c105150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 --2- 172.21.15.19:0/187566401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c103b00 0x7fe18c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 -- 172.21.15.19:0/187566401 >> 172.21.15.19:0/187566401 conn(0x7fe18c0fed10 msgr2=0x7fe18c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 -- 172.21.15.19:0/187566401 shutdown_connections 2024-04-25T10:52:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 -- 172.21.15.19:0/187566401 wait complete. 2024-04-25T10:52:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.897+0000 7fe191af2640 1 Processor -- start 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe191af2640 1 -- start start 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe191af2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103b00 0x7fe18c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe191af2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18c104d40 0x7fe18c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe191af2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c109a40 0x7fe18c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe191af2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10f600 con 0x7fe18c103b00 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe191af2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10f480 con 0x7fe18c109a40 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe191af2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10f780 con 0x7fe18c104d40 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103b00 0x7fe18c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103b00 0x7fe18c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40204/0 (socket says 172.21.15.19:40204) 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 -- 172.21.15.19:0/2753274382 learned_addr learned my addr 172.21.15.19:0/2753274382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18bfff640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c109a40 0x7fe18c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18affd640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18c104d40 0x7fe18c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 -- 172.21.15.19:0/2753274382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18c104d40 msgr2=0x7fe18c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18c104d40 0x7fe18c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 -- 172.21.15.19:0/2753274382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c109a40 msgr2=0x7fe18c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c109a40 0x7fe18c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18b7fe640 1 -- 172.21.15.19:0/2753274382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c1a1cc0 con 0x7fe18c103b00 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.898+0000 7fe18bfff640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c109a40 0x7fe18c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.899+0000 7fe18b7fe640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103b00 0x7fe18c19a230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00e9f0 tx=0x7fe17c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.899+0000 7fe188ff9640 1 -- 172.21.15.19:0/2753274382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c00ccc0 con 0x7fe18c103b00 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.899+0000 7fe188ff9640 1 -- 172.21.15.19:0/2753274382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17c0035f0 con 0x7fe18c103b00 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.899+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c1a1fb0 con 0x7fe18c103b00 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.899+0000 7fe188ff9640 1 -- 172.21.15.19:0/2753274382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c00ce60 con 0x7fe18c103b00 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.899+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c1a2410 con 0x7fe18c103b00 2024-04-25T10:52:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.900+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c06a1c0 con 0x7fe18c103b00 2024-04-25T10:52:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.902+0000 7fe188ff9640 1 -- 172.21.15.19:0/2753274382 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe17c003aa0 con 0x7fe18c103b00 2024-04-25T10:52:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.903+0000 7fe188ff9640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1740afaf0 0x7fe1740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.903+0000 7fe18affd640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1740afaf0 0x7fe1740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.903+0000 7fe188ff9640 1 -- 172.21.15.19:0/2753274382 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe17c0dec70 con 0x7fe18c103b00 2024-04-25T10:52:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.903+0000 7fe18affd640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1740afaf0 0x7fe1740b2000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe180031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:13.906+0000 7fe188ff9640 1 -- 172.21.15.19:0/2753274382 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe18c06a1c0 con 0x7fe18c103b00 2024-04-25T10:52:14.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.134+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe18c02dc10 con 0x7fe18c103b00 2024-04-25T10:52:14.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.135+0000 7fe188ff9640 1 -- 172.21.15.19:0/2753274382 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7fe17c0a7c50 con 0x7fe18c103b00 2024-04-25T10:52:14.136 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:14.136 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T10:52:14.136 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1740afaf0 msgr2=0x7fe1740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1740afaf0 0x7fe1740b2000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe180031400 comp rx=0 tx=0).stop 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103b00 msgr2=0x7fe18c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103b00 0x7fe18c19a230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00e9f0 tx=0x7fe17c00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 shutdown_connections 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c109a40 0x7fe18c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18c104d40 0x7fe18c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1740afaf0 0x7fe1740b2000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 --2- 172.21.15.19:0/2753274382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103b00 0x7fe18c19a230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 >> 172.21.15.19:0/2753274382 conn(0x7fe18c0fed10 msgr2=0x7fe18c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 shutdown_connections 2024-04-25T10:52:14.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:14.137+0000 7fe191af2640 1 -- 172.21.15.19:0/2753274382 wait complete. 2024-04-25T10:52:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:14 smithi169 ceph-mon[33049]: Deploying daemon mds.h on smithi112 2024-04-25T10:52:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:14 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.169:6836/717205661,v1:172.21.15.169:6837/717205661] up:boot 2024-04-25T10:52:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:14 smithi169 ceph-mon[33049]: fsmap 7 up:standby 2024-04-25T10:52:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:14 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:52:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:14 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2753274382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:14.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:14 smithi112 ceph-mon[32799]: Deploying daemon mds.h on smithi112 2024-04-25T10:52:14.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:14 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.169:6836/717205661,v1:172.21.15.169:6837/717205661] up:boot 2024-04-25T10:52:14.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:14 smithi112 ceph-mon[32799]: fsmap 7 up:standby 2024-04-25T10:52:14.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:14 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:52:14.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:14 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2753274382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:14.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:14 smithi019 ceph-mon[29209]: Deploying daemon mds.h on smithi112 2024-04-25T10:52:14.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:14 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.169:6836/717205661,v1:172.21.15.169:6837/717205661] up:boot 2024-04-25T10:52:14.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:14 smithi019 ceph-mon[29209]: fsmap 7 up:standby 2024-04-25T10:52:14.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:14 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:52:14.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:14 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2753274382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:15.416 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 systemd[1]: Starting Ceph mds.h for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:15.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:15 smithi169 ceph-mon[33049]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:15.756 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:15 smithi112 ceph-mon[32799]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:15.756 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 podman[62259]: 2024-04-25 10:52:15.405216076 +0000 UTC m=+0.018028109 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:15.756 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 podman[62259]: 2024-04-25 10:52:15.528788033 +0000 UTC m=+0.141600039 container create ffa8dbdcee342ade478faccd0b6e12622150ae56c61c7e26fa5446f40e4c4c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T10:52:15.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:15 smithi019 ceph-mon[29209]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:16.059 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 podman[62259]: 2024-04-25 10:52:15.756279947 +0000 UTC m=+0.369091983 container init ffa8dbdcee342ade478faccd0b6e12622150ae56c61c7e26fa5446f40e4c4c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:52:16.059 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 podman[62259]: 2024-04-25 10:52:15.759099094 +0000 UTC m=+0.371911100 container start ffa8dbdcee342ade478faccd0b6e12622150ae56c61c7e26fa5446f40e4c4c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:52:16.059 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 ceph-mds[62279]: -- 172.21.15.112:0/1120248520 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556267a63a40 con 0x556267aaba80 2024-04-25T10:52:16.059 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h[62275]: starting mds.h at 2024-04-25T10:52:16.059 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 bash[62259]: ffa8dbdcee342ade478faccd0b6e12622150ae56c61c7e26fa5446f40e4c4c47 2024-04-25T10:52:16.059 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 10:52:15 smithi112 systemd[1]: Started Ceph mds.h for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:16.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:16.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 -- 172.21.15.19:0/2489823406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 msgr2=0x7f1298104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:16.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 --2- 172.21.15.19:0/2489823406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f1298104030 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f128c009900 tx=0x7f128c02a410 comp rx=0 tx=0).stop 2024-04-25T10:52:16.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 -- 172.21.15.19:0/2489823406 shutdown_connections 2024-04-25T10:52:16.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 --2- 172.21.15.19:0/2489823406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1298108920 0x7f129810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 --2- 172.21.15.19:0/2489823406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f1298104030 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 --2- 172.21.15.19:0/2489823406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12981029e0 0x7f12981036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 -- 172.21.15.19:0/2489823406 >> 172.21.15.19:0/2489823406 conn(0x7f12980fdbd0 msgr2=0x7f1298100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 -- 172.21.15.19:0/2489823406 shutdown_connections 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.276+0000 7f129d55f640 1 -- 172.21.15.19:0/2489823406 wait complete. 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 Processor -- start 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 -- start start 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12981029e0 0x7f1298075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f129819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1298108920 0x7f12981a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f129810e4e0 con 0x7f1298103c20 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f129810e360 con 0x7f1298108920 2024-04-25T10:52:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f129d55f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f129810e660 con 0x7f12981029e0 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f12967fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f129819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f12967fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f129819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40224/0 (socket says 172.21.15.19:40224) 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f12967fc640 1 -- 172.21.15.19:0/1213940719 learned_addr learned my addr 172.21.15.19:0/1213940719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f1296ffd640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12981029e0 0x7f1298075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.277+0000 7f12977fe640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1298108920 0x7f12981a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f12967fc640 1 -- 172.21.15.19:0/1213940719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12981029e0 msgr2=0x7f1298075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f12967fc640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12981029e0 0x7f1298075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f12967fc640 1 -- 172.21.15.19:0/1213940719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1298108920 msgr2=0x7f12981a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f12967fc640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1298108920 0x7f12981a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f12967fc640 1 -- 172.21.15.19:0/1213940719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12981a1d40 con 0x7f1298103c20 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f12967fc640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f129819efb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f128c006010 tx=0x7f128c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f127ffff640 1 -- 172.21.15.19:0/1213940719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c03f030 con 0x7f1298103c20 2024-04-25T10:52:16.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12981a1fd0 con 0x7f1298103c20 2024-04-25T10:52:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f127ffff640 1 -- 172.21.15.19:0/1213940719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128c031910 con 0x7f1298103c20 2024-04-25T10:52:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f127ffff640 1 -- 172.21.15.19:0/1213940719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c035070 con 0x7f1298103c20 2024-04-25T10:52:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.278+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981a2440 con 0x7f1298103c20 2024-04-25T10:52:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.279+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1298107730 con 0x7f1298103c20 2024-04-25T10:52:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.281+0000 7f127ffff640 1 -- 172.21.15.19:0/1213940719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f128c042030 con 0x7f1298103c20 2024-04-25T10:52:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.282+0000 7f127ffff640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12800afaf0 0x7f12800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.282+0000 7f1296ffd640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12800afaf0 0x7f12800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.282+0000 7f127ffff640 1 -- 172.21.15.19:0/1213940719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f128c0feed0 con 0x7f1298103c20 2024-04-25T10:52:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.283+0000 7f1296ffd640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12800afaf0 0x7f12800b2000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12981035e0 tx=0x7f1288005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:16.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.286+0000 7f127ffff640 1 -- 172.21.15.19:0/1213940719 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f128c0c7860 con 0x7f1298103c20 2024-04-25T10:52:16.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.516+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f129802db50 con 0x7f1298103c20 2024-04-25T10:52:16.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.516+0000 7f127ffff640 1 -- 172.21.15.19:0/1213940719 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f128c0cbb60 con 0x7f1298103c20 2024-04-25T10:52:16.517 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:16.517 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T10:52:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T10:52:16.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12800afaf0 msgr2=0x7f12800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:16.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12800afaf0 0x7f12800b2000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12981035e0 tx=0x7f1288005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 msgr2=0x7f129819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f129819efb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f128c006010 tx=0x7f128c0042f0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 shutdown_connections 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1298108920 0x7f12981a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f12800afaf0 0x7f12800b2000 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1298103c20 0x7f129819efb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 --2- 172.21.15.19:0/1213940719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12981029e0 0x7f1298075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 >> 172.21.15.19:0/1213940719 conn(0x7f12980fdbd0 msgr2=0x7f1298104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.518+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 shutdown_connections 2024-04-25T10:52:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:16.519+0000 7f129d55f640 1 -- 172.21.15.19:0/1213940719 wait complete. 2024-04-25T10:52:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: Deploying daemon mds.i on smithi169 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.112:6836/820120755,v1:172.21.15.112:6837/820120755] up:boot 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: fsmap 8 up:standby 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:52:17.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:16 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1213940719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: Deploying daemon mds.i on smithi169 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.112:6836/820120755,v1:172.21.15.112:6837/820120755] up:boot 2024-04-25T10:52:17.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: fsmap 8 up:standby 2024-04-25T10:52:17.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:52:17.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:16 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1213940719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: Deploying daemon mds.i on smithi169 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.112:6836/820120755,v1:172.21.15.112:6837/820120755] up:boot 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: fsmap 8 up:standby 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:52:17.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:16 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1213940719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:18.232 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 systemd[1]: Starting Ceph mds.i for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:18.484 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 podman[63894]: 2024-04-25 10:52:18.231430983 +0000 UTC m=+0.110360899 container create 4dcb3f51c65282f8eeea4eae4f60712fc8752f2601c71d6ce7552fd628ccb7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T10:52:18.484 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 podman[63894]: 2024-04-25 10:52:18.139267848 +0000 UTC m=+0.018197729 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:18.484 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 podman[63894]: 2024-04-25 10:52:18.483885924 +0000 UTC m=+0.362815799 container init 4dcb3f51c65282f8eeea4eae4f60712fc8752f2601c71d6ce7552fd628ccb7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:52:18.532 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:18.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.658+0000 7ff3805c0640 1 -- 172.21.15.19:0/1984515103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781080a0 msgr2=0x7ff37810a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.658+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1984515103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781080a0 0x7ff37810a4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff370009ca0 tx=0x7ff37002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.658+0000 7ff3805c0640 1 -- 172.21.15.19:0/1984515103 shutdown_connections 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.658+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1984515103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781080a0 0x7ff37810a4c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.658+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1984515103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3781033a0 0x7ff3781037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.658+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1984515103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff378102160 0x7ff378102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.658+0000 7ff3805c0640 1 -- 172.21.15.19:0/1984515103 >> 172.21.15.19:0/1984515103 conn(0x7ff3780fd330 msgr2=0x7ff3780ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.659+0000 7ff3805c0640 1 -- 172.21.15.19:0/1984515103 shutdown_connections 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.659+0000 7ff3805c0640 1 -- 172.21.15.19:0/1984515103 wait complete. 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.659+0000 7ff3805c0640 1 Processor -- start 2024-04-25T10:52:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.659+0000 7ff3805c0640 1 -- start start 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff3805c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff378102160 0x7ff37819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff3805c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781033a0 0x7ff37819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff3805c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3781080a0 0x7ff3781a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff3805c0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff37810d440 con 0x7ff3781033a0 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff3805c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff37810d2c0 con 0x7ff3781080a0 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff3805c0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff37810d5c0 con 0x7ff378102160 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781033a0 0x7ff37819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781033a0 0x7ff37819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40242/0 (socket says 172.21.15.19:40242) 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 -- 172.21.15.19:0/1103779314 learned_addr learned my addr 172.21.15.19:0/1103779314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37e35c640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff378102160 0x7ff37819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37eb5d640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3781080a0 0x7ff3781a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 -- 172.21.15.19:0/1103779314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff378102160 msgr2=0x7ff37819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff378102160 0x7ff37819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 -- 172.21.15.19:0/1103779314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3781080a0 msgr2=0x7ff3781a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3781080a0 0x7ff3781a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 -- 172.21.15.19:0/1103779314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3781a1d50 con 0x7ff3781033a0 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff37db5b640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781033a0 0x7ff37819a750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff37400e9f0 tx=0x7ff37400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.660+0000 7ff36b7fe640 1 -- 172.21.15.19:0/1103779314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37400cd80 con 0x7ff3781033a0 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.661+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3781a1fe0 con 0x7ff3781033a0 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.661+0000 7ff36b7fe640 1 -- 172.21.15.19:0/1103779314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3740054d0 con 0x7ff3781033a0 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.661+0000 7ff36b7fe640 1 -- 172.21.15.19:0/1103779314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374010690 con 0x7ff3781033a0 2024-04-25T10:52:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.661+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3781a2480 con 0x7ff3781033a0 2024-04-25T10:52:18.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.662+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3781a2740 con 0x7ff3781033a0 2024-04-25T10:52:18.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.663+0000 7ff36b7fe640 1 -- 172.21.15.19:0/1103779314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff374004e90 con 0x7ff3781033a0 2024-04-25T10:52:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.664+0000 7ff36b7fe640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3640afaf0 0x7ff3640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.664+0000 7ff37e35c640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3640afaf0 0x7ff3640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.664+0000 7ff36b7fe640 1 -- 172.21.15.19:0/1103779314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ff3740dec20 con 0x7ff3781033a0 2024-04-25T10:52:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.664+0000 7ff37e35c640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3640afaf0 0x7ff3640b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff36c005fd0 tx=0x7ff36c00b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.669+0000 7ff36b7fe640 1 -- 172.21.15.19:0/1103779314 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3740a7c00 con 0x7ff3781033a0 2024-04-25T10:52:18.751 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 podman[63894]: 2024-04-25 10:52:18.486573703 +0000 UTC m=+0.365503579 container start 4dcb3f51c65282f8eeea4eae4f60712fc8752f2601c71d6ce7552fd628ccb7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-25T10:52:18.751 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 ceph-mds[63913]: -- 172.21.15.169:0/2206787587 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f411225a40 con 0x55f41126d180 2024-04-25T10:52:18.751 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i[63909]: starting mds.i at 2024-04-25T10:52:18.751 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 bash[63894]: 4dcb3f51c65282f8eeea4eae4f60712fc8752f2601c71d6ce7552fd628ccb7d4 2024-04-25T10:52:18.752 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 10:52:18 smithi169 systemd[1]: Started Ceph mds.i for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:18.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.902+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3781a2910 con 0x7ff3781033a0 2024-04-25T10:52:18.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.902+0000 7ff36b7fe640 1 -- 172.21.15.19:0/1103779314 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7ff3740a7500 con 0x7ff3781033a0 2024-04-25T10:52:18.904 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:18.904 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T10:52:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3640afaf0 msgr2=0x7ff3640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3640afaf0 0x7ff3640b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff36c005fd0 tx=0x7ff36c00b510 comp rx=0 tx=0).stop 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781033a0 msgr2=0x7ff37819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781033a0 0x7ff37819a750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff37400e9f0 tx=0x7ff37400edf0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 shutdown_connections 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3781080a0 0x7ff3781a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3640afaf0 0x7ff3640b2000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3781033a0 0x7ff37819a750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 --2- 172.21.15.19:0/1103779314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff378102160 0x7ff37819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 >> 172.21.15.19:0/1103779314 conn(0x7ff3780fd330 msgr2=0x7ff378106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 shutdown_connections 2024-04-25T10:52:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:18.905+0000 7ff3805c0640 1 -- 172.21.15.19:0/1103779314 wait complete. 2024-04-25T10:52:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:19.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: Deploying daemon mds.g on smithi019 2024-04-25T10:52:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1103779314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.169:6838/386384439,v1:172.21.15.169:6839/386384439] up:boot 2024-04-25T10:52:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: fsmap 9 up:standby 2024-04-25T10:52:19.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:19 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:52:20.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:20.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:20.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:20.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: Deploying daemon mds.g on smithi019 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1103779314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.169:6838/386384439,v1:172.21.15.169:6839/386384439] up:boot 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: fsmap 9 up:standby 2024-04-25T10:52:20.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:19 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: Deploying daemon mds.g on smithi019 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1103779314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.169:6838/386384439,v1:172.21.15.169:6839/386384439] up:boot 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: fsmap 9 up:standby 2024-04-25T10:52:20.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:19 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:52:20.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:20.944 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:20 smithi019 systemd[1]: Starting Ceph mds.g for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:21.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 -- 172.21.15.19:0/4160283050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20107680 msgr2=0x7fba20107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 --2- 172.21.15.19:0/4160283050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20107680 0x7fba20107a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba18009ca0 tx=0x7fba1802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 -- 172.21.15.19:0/4160283050 shutdown_connections 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 --2- 172.21.15.19:0/4160283050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20107680 0x7fba20107a90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 --2- 172.21.15.19:0/4160283050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba20104d20 0x7fba20107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 --2- 172.21.15.19:0/4160283050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba201023c0 0x7fba201047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 -- 172.21.15.19:0/4160283050 >> 172.21.15.19:0/4160283050 conn(0x7fba200fdbf0 msgr2=0x7fba20100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 -- 172.21.15.19:0/4160283050 shutdown_connections 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.042+0000 7fba27914640 1 -- 172.21.15.19:0/4160283050 wait complete. 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 Processor -- start 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 -- start start 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba201023c0 0x7fba201a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20104d20 0x7fba201a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba20107680 0x7fba201a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba20115c00 con 0x7fba20104d20 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba20115a80 con 0x7fba201023c0 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba27914640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba20115d80 con 0x7fba20107680 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba24eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20104d20 0x7fba201a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba24eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20104d20 0x7fba201a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40254/0 (socket says 172.21.15.19:40254) 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba24eaf640 1 -- 172.21.15.19:0/2852073846 learned_addr learned my addr 172.21.15.19:0/2852073846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba256b0640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba201023c0 0x7fba201a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.043+0000 7fba25eb1640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba20107680 0x7fba201a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba24eaf640 1 -- 172.21.15.19:0/2852073846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba20107680 msgr2=0x7fba201a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba24eaf640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba20107680 0x7fba201a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba24eaf640 1 -- 172.21.15.19:0/2852073846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba201023c0 msgr2=0x7fba201a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba24eaf640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba201023c0 0x7fba201a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba24eaf640 1 -- 172.21.15.19:0/2852073846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba201a8350 con 0x7fba20104d20 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba24eaf640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20104d20 0x7fba201a0d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fba1000e9f0 tx=0x7fba1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba167fc640 1 -- 172.21.15.19:0/2852073846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1000cd80 con 0x7fba20104d20 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba167fc640 1 -- 172.21.15.19:0/2852073846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba100108e0 con 0x7fba20104d20 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba167fc640 1 -- 172.21.15.19:0/2852073846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10018980 con 0x7fba20104d20 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba201a8640 con 0x7fba20104d20 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.044+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba201a8b90 con 0x7fba20104d20 2024-04-25T10:52:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.045+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba200ff710 con 0x7fba20104d20 2024-04-25T10:52:21.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.047+0000 7fba167fc640 1 -- 172.21.15.19:0/2852073846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fba10010430 con 0x7fba20104d20 2024-04-25T10:52:21.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.047+0000 7fba167fc640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fba0c0afaf0 0x7fba0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:21.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.047+0000 7fba167fc640 1 -- 172.21.15.19:0/2852073846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fba100defc0 con 0x7fba20104d20 2024-04-25T10:52:21.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.047+0000 7fba256b0640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fba0c0afaf0 0x7fba0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:21.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.048+0000 7fba256b0640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fba0c0afaf0 0x7fba0c0b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba200757f0 tx=0x7fba1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:21.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.051+0000 7fba167fc640 1 -- 172.21.15.19:0/2852073846 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba100a8040 con 0x7fba20104d20 2024-04-25T10:52:21.226 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 podman[70896]: 2024-04-25 10:52:21.001916597 +0000 UTC m=+0.019079520 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:21.226 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 podman[70896]: 2024-04-25 10:52:21.125006915 +0000 UTC m=+0.142169818 container create 1ca4506a6f3aebb7c6e8ef3d924901706207a649c29b9ec6d7148e73dab65759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T10:52:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.282+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba2002db50 con 0x7fba20104d20 2024-04-25T10:52:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.283+0000 7fba167fc640 1 -- 172.21.15.19:0/2852073846 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7fba100a7940 con 0x7fba20104d20 2024-04-25T10:52:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:21.284 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T10:52:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fba0c0afaf0 msgr2=0x7fba0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fba0c0afaf0 0x7fba0c0b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba200757f0 tx=0x7fba1c005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20104d20 msgr2=0x7fba201a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20104d20 0x7fba201a0d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fba1000e9f0 tx=0x7fba1000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 shutdown_connections 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba20107680 0x7fba201a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fba0c0afaf0 0x7fba0c0b2000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba20104d20 0x7fba201a0d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 --2- 172.21.15.19:0/2852073846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba201023c0 0x7fba201a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.285+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 >> 172.21.15.19:0/2852073846 conn(0x7fba200fdbf0 msgr2=0x7fba20103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.286+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 shutdown_connections 2024-04-25T10:52:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:21.286+0000 7fba27914640 1 -- 172.21.15.19:0/2852073846 wait complete. 2024-04-25T10:52:21.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 podman[70896]: 2024-04-25 10:52:21.336135106 +0000 UTC m=+0.353298025 container init 1ca4506a6f3aebb7c6e8ef3d924901706207a649c29b9ec6d7148e73dab65759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-25T10:52:21.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 podman[70896]: 2024-04-25 10:52:21.339066457 +0000 UTC m=+0.356229371 container start 1ca4506a6f3aebb7c6e8ef3d924901706207a649c29b9ec6d7148e73dab65759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T10:52:21.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mds[70936]: -- 172.21.15.19:0/4187699372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561cb1983a40 con 0x561cb19cb600 2024-04-25T10:52:21.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g[70930]: starting mds.g at 2024-04-25T10:52:21.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 bash[70896]: 1ca4506a6f3aebb7c6e8ef3d924901706207a649c29b9ec6d7148e73dab65759 2024-04-25T10:52:21.612 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 10:52:21 smithi019 systemd[1]: Started Ceph mds.g for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:21.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:21.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2852073846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:21.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:21.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:21.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:21.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:21.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:21.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:21 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:22.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:22.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2852073846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:22.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:22.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:22.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:22.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:22.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:22.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:21 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2852073846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:22.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:21 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:22.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:22 smithi019 ceph-mon[29209]: Deploying daemon mds.a on smithi019 2024-04-25T10:52:22.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:22 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.19:6838/154007495,v1:172.21.15.19:6839/154007495] up:boot 2024-04-25T10:52:22.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:22 smithi019 ceph-mon[29209]: fsmap 10 up:standby 2024-04-25T10:52:22.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:22 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:52:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:22 smithi169 ceph-mon[33049]: Deploying daemon mds.a on smithi019 2024-04-25T10:52:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:22 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.19:6838/154007495,v1:172.21.15.19:6839/154007495] up:boot 2024-04-25T10:52:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:22 smithi169 ceph-mon[33049]: fsmap 10 up:standby 2024-04-25T10:52:22.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:22 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:52:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:22 smithi112 ceph-mon[32799]: Deploying daemon mds.a on smithi019 2024-04-25T10:52:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:22 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.19:6838/154007495,v1:172.21.15.19:6839/154007495] up:boot 2024-04-25T10:52:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:22 smithi112 ceph-mon[32799]: fsmap 10 up:standby 2024-04-25T10:52:22.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:22 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:52:23.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.516+0000 7f40e4357640 1 -- 172.21.15.19:0/3571131448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc106c20 msgr2=0x7f40dc1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.516+0000 7f40e4357640 1 --2- 172.21.15.19:0/3571131448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc106c20 0x7f40dc1090c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009900 tx=0x7f40cc02a3f0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 -- 172.21.15.19:0/3571131448 shutdown_connections 2024-04-25T10:52:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 --2- 172.21.15.19:0/3571131448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc109750 0x7f40dc10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 --2- 172.21.15.19:0/3571131448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc106c20 0x7f40dc1090c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 --2- 172.21.15.19:0/3571131448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40dc104230 0x7f40dc106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 -- 172.21.15.19:0/3571131448 >> 172.21.15.19:0/3571131448 conn(0x7f40dc0fdbd0 msgr2=0x7f40dc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 -- 172.21.15.19:0/3571131448 shutdown_connections 2024-04-25T10:52:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 -- 172.21.15.19:0/3571131448 wait complete. 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.517+0000 7f40e4357640 1 Processor -- start 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e4357640 1 -- start start 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e4357640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40dc104230 0x7f40dc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e4357640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc106c20 0x7f40dc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e4357640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc109750 0x7f40dc1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e4357640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10f610 con 0x7f40dc109750 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e4357640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10f490 con 0x7f40dc106c20 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e4357640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10f790 con 0x7f40dc104230 2024-04-25T10:52:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc109750 0x7f40dc1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc109750 0x7f40dc1a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59348/0 (socket says 172.21.15.19:59348) 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e20f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40dc104230 0x7f40dc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 -- 172.21.15.19:0/2839001247 learned_addr learned my addr 172.21.15.19:0/2839001247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e18f2640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc106c20 0x7f40dc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 -- 172.21.15.19:0/2839001247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40dc104230 msgr2=0x7f40dc19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40dc104230 0x7f40dc19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 -- 172.21.15.19:0/2839001247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc106c20 msgr2=0x7f40dc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc106c20 0x7f40dc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.518+0000 7f40e28f4640 1 -- 172.21.15.19:0/2839001247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc1a1c40 con 0x7f40dc109750 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.519+0000 7f40e28f4640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc109750 0x7f40dc1a1680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40d400cce0 tx=0x7f40d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.519+0000 7f40d37fe640 1 -- 172.21.15.19:0/2839001247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d400f450 con 0x7f40dc109750 2024-04-25T10:52:23.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.519+0000 7f40d37fe640 1 -- 172.21.15.19:0/2839001247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d400faa0 con 0x7f40dc109750 2024-04-25T10:52:23.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.519+0000 7f40d37fe640 1 -- 172.21.15.19:0/2839001247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4018a80 con 0x7f40dc109750 2024-04-25T10:52:23.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.519+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc1a1f30 con 0x7f40dc109750 2024-04-25T10:52:23.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.519+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc1a2390 con 0x7f40dc109750 2024-04-25T10:52:23.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.520+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc0ff6f0 con 0x7f40dc109750 2024-04-25T10:52:23.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.521+0000 7f40d37fe640 1 -- 172.21.15.19:0/2839001247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f40d400fc50 con 0x7f40dc109750 2024-04-25T10:52:23.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.522+0000 7f40d37fe640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c80afb70 0x7f40c80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:23.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.523+0000 7f40e20f3640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c80afb70 0x7f40c80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:23.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.523+0000 7f40d37fe640 1 -- 172.21.15.19:0/2839001247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f40d40deb80 con 0x7f40dc109750 2024-04-25T10:52:23.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.523+0000 7f40e20f3640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c80afb70 0x7f40c80b2080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40dc06a810 tx=0x7f40d800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:23.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.527+0000 7f40d37fe640 1 -- 172.21.15.19:0/2839001247 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40d4014030 con 0x7f40dc109750 2024-04-25T10:52:23.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.765+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40dc0637d0 con 0x7f40dc109750 2024-04-25T10:52:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.767+0000 7f40d37fe640 1 -- 172.21.15.19:0/2839001247 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7f40d40a7b60 con 0x7f40dc109750 2024-04-25T10:52:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:23.768 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T10:52:23.768 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c80afb70 msgr2=0x7f40c80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c80afb70 0x7f40c80b2080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40dc06a810 tx=0x7f40d800a2d0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc109750 msgr2=0x7f40dc1a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc109750 0x7f40dc1a1680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40d400cce0 tx=0x7f40d400a620 comp rx=0 tx=0).stop 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 shutdown_connections 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f40c80afb70 0x7f40c80b2080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc109750 0x7f40dc1a1680 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc106c20 0x7f40dc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 --2- 172.21.15.19:0/2839001247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40dc104230 0x7f40dc19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 >> 172.21.15.19:0/2839001247 conn(0x7f40dc0fdbd0 msgr2=0x7f40dc105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 shutdown_connections 2024-04-25T10:52:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:23.769+0000 7f40e4357640 1 -- 172.21.15.19:0/2839001247 wait complete. 2024-04-25T10:52:23.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:23 smithi019 ceph-mon[29209]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:24.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:23 smithi169 ceph-mon[33049]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:24.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:23 smithi112 ceph-mon[32799]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:24.611 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 10:52:24 smithi019 systemd[1]: Started Ceph mds.a for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:24.949 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:24 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2839001247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:24.949 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:24.949 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:24.949 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:24.949 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:24.949 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:24.949 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:24 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:24 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2839001247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:25.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:24 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2839001247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:52:25.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:52:25.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:24 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:25.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.912+0000 7fca1735e640 1 -- 172.21.15.19:0/3551861045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 msgr2=0x7fca1010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.912+0000 7fca1735e640 1 --2- 172.21.15.19:0/3551861045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca1010bcb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fca08009ca0 tx=0x7fca0802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 -- 172.21.15.19:0/3551861045 shutdown_connections 2024-04-25T10:52:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 --2- 172.21.15.19:0/3551861045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca1010bcb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 --2- 172.21.15.19:0/3551861045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca10106f30 0x7fca10109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 --2- 172.21.15.19:0/3551861045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca101045d0 0x7fca101069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 -- 172.21.15.19:0/3551861045 >> 172.21.15.19:0/3551861045 conn(0x7fca100fdbd0 msgr2=0x7fca10100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 -- 172.21.15.19:0/3551861045 shutdown_connections 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 -- 172.21.15.19:0/3551861045 wait complete. 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 Processor -- start 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.913+0000 7fca1735e640 1 -- start start 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca1735e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca101045d0 0x7fca1019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca1735e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca10106f30 0x7fca1019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca1735e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca101a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca1735e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca1010f5f0 con 0x7fca10109890 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca1735e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca1010f470 con 0x7fca101045d0 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca1735e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca1010f770 con 0x7fca10106f30 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca101a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca150fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca101045d0 0x7fca1019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca101a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59396/0 (socket says 172.21.15.19:59396) 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 -- 172.21.15.19:0/594404420 learned_addr learned my addr 172.21.15.19:0/594404420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca148f9640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca10106f30 0x7fca1019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 -- 172.21.15.19:0/594404420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca10106f30 msgr2=0x7fca1019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca10106f30 0x7fca1019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 -- 172.21.15.19:0/594404420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca101045d0 msgr2=0x7fca1019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca101045d0 0x7fca1019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 -- 172.21.15.19:0/594404420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca101a1d30 con 0x7fca10109890 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca150fa640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca101045d0 0x7fca1019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca158fb640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca101a16c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca08009430 tx=0x7fca080319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:25.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.914+0000 7fca067fc640 1 -- 172.21.15.19:0/594404420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08003b70 con 0x7fca10109890 2024-04-25T10:52:25.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.915+0000 7fca067fc640 1 -- 172.21.15.19:0/594404420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca08003d10 con 0x7fca10109890 2024-04-25T10:52:25.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.915+0000 7fca067fc640 1 -- 172.21.15.19:0/594404420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0802fb30 con 0x7fca10109890 2024-04-25T10:52:25.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.915+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca101a1fc0 con 0x7fca10109890 2024-04-25T10:52:25.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.915+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca101a2480 con 0x7fca10109890 2024-04-25T10:52:25.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.916+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca100ff6f0 con 0x7fca10109890 2024-04-25T10:52:25.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.917+0000 7fca067fc640 1 -- 172.21.15.19:0/594404420 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fca0802fcd0 con 0x7fca10109890 2024-04-25T10:52:25.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.918+0000 7fca067fc640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9fc0afb70 0x7fc9fc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:25.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.918+0000 7fca150fa640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9fc0afb70 0x7fc9fc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:25.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.918+0000 7fca067fc640 1 -- 172.21.15.19:0/594404420 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fca080ffc70 con 0x7fca10109890 2024-04-25T10:52:25.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.919+0000 7fca150fa640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9fc0afb70 0x7fc9fc0b2080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca100757f0 tx=0x7fca0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:25.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:25.923+0000 7fca067fc640 1 -- 172.21.15.19:0/594404420 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca080c8c50 con 0x7fca10109890 2024-04-25T10:52:25.980 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:25 smithi112 ceph-mon[32799]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:25.981 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:25 smithi112 ceph-mon[32799]: Deploying daemon mds.b on smithi112 2024-04-25T10:52:25.981 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:25 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.19:6840/427683873,v1:172.21.15.19:6841/427683873] up:boot 2024-04-25T10:52:25.981 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:25 smithi112 ceph-mon[32799]: fsmap 11 up:standby 2024-04-25T10:52:25.981 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:25 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:52:26.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:25 smithi169 ceph-mon[33049]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:26.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:25 smithi169 ceph-mon[33049]: Deploying daemon mds.b on smithi112 2024-04-25T10:52:26.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:25 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.19:6840/427683873,v1:172.21.15.19:6841/427683873] up:boot 2024-04-25T10:52:26.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:25 smithi169 ceph-mon[33049]: fsmap 11 up:standby 2024-04-25T10:52:26.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:25 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:52:26.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:25 smithi019 ceph-mon[29209]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:26.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:25 smithi019 ceph-mon[29209]: Deploying daemon mds.b on smithi112 2024-04-25T10:52:26.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:25 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.19:6840/427683873,v1:172.21.15.19:6841/427683873] up:boot 2024-04-25T10:52:26.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:25 smithi019 ceph-mon[29209]: fsmap 11 up:standby 2024-04-25T10:52:26.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:25 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:52:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.150+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca1002db50 con 0x7fca10109890 2024-04-25T10:52:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.151+0000 7fca067fc640 1 -- 172.21.15.19:0/594404420 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7fca080c8550 con 0x7fca10109890 2024-04-25T10:52:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:26.152 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T10:52:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 12 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9fc0afb70 msgr2=0x7fc9fc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9fc0afb70 0x7fc9fc0b2080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca100757f0 tx=0x7fca0c005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 msgr2=0x7fca101a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca101a16c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca08009430 tx=0x7fca080319f0 comp rx=0 tx=0).stop 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 shutdown_connections 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc9fc0afb70 0x7fc9fc0b2080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca10109890 0x7fca101a16c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca10106f30 0x7fca1019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 --2- 172.21.15.19:0/594404420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca101045d0 0x7fca1019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 >> 172.21.15.19:0/594404420 conn(0x7fca100fdbd0 msgr2=0x7fca101051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:26.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 shutdown_connections 2024-04-25T10:52:26.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:26.153+0000 7fca1735e640 1 -- 172.21.15.19:0/594404420 wait complete. 2024-04-25T10:52:26.559 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:26 smithi112 systemd[1]: Starting Ceph mds.b for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T10:52:26.942 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:26 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/594404420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:26.943 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:26 smithi112 podman[62572]: 2024-04-25 10:52:26.596257073 +0000 UTC m=+0.018151168 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:52:26.943 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:26 smithi112 podman[62572]: 2024-04-25 10:52:26.698635616 +0000 UTC m=+0.120529745 container create 902bbc03c6c421415831d11331a2893c16adf553c8567c8d360e0f95dcf16675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T10:52:26.943 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:26 smithi112 podman[62572]: 2024-04-25 10:52:26.942709382 +0000 UTC m=+0.364603465 container init 902bbc03c6c421415831d11331a2893c16adf553c8567c8d360e0f95dcf16675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T10:52:27.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:26 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/594404420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:27.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:26 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/594404420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:27.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:26 smithi112 podman[62572]: 2024-04-25 10:52:26.945311297 +0000 UTC m=+0.367205380 container start 902bbc03c6c421415831d11331a2893c16adf553c8567c8d360e0f95dcf16675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:52:27.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:26 smithi112 ceph-mds[62592]: -- 172.21.15.112:0/3383929007 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9f55c1a40 con 0x55e9f5609180 2024-04-25T10:52:27.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b[62588]: starting mds.b at 2024-04-25T10:52:27.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:27 smithi112 bash[62572]: 902bbc03c6c421415831d11331a2893c16adf553c8567c8d360e0f95dcf16675 2024-04-25T10:52:27.310 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:27 smithi112 systemd[1]: Started Ceph mds.b for 9e90b6fe-02f0-11ef-bc93-c7b262605968. 2024-04-25T10:52:28.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:27 smithi169 ceph-mon[33049]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:28.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:27 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:52:28.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:27 smithi112 ceph-mon[32799]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:28.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:27 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:52:28.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:27 smithi019 ceph-mon[29209]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:28.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:28.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:27 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:52:28.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 -- 172.21.15.19:0/2953235611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 msgr2=0x7fe240104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 --2- 172.21.15.19:0/2953235611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe240104010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009900 tx=0x7fe23c02a410 comp rx=0 tx=0).stop 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 -- 172.21.15.19:0/2953235611 shutdown_connections 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 --2- 172.21.15.19:0/2953235611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe240108900 0x7fe24010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 --2- 172.21.15.19:0/2953235611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe240104010 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 --2- 172.21.15.19:0/2953235611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2401029c0 0x7fe2401036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 -- 172.21.15.19:0/2953235611 >> 172.21.15.19:0/2953235611 conn(0x7fe2400fdbd0 msgr2=0x7fe240100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 -- 172.21.15.19:0/2953235611 shutdown_connections 2024-04-25T10:52:28.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 -- 172.21.15.19:0/2953235611 wait complete. 2024-04-25T10:52:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.282+0000 7fe247c92640 1 Processor -- start 2024-04-25T10:52:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe247c92640 1 -- start start 2024-04-25T10:52:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe247c92640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2401029c0 0x7fe24019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe247c92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe24019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe247c92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe240108900 0x7fe2401a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe247c92640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe24010e4f0 con 0x7fe240103c00 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe247c92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe24010e370 con 0x7fe240108900 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe247c92640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe24010e670 con 0x7fe2401029c0 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe24019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe245a2e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2401029c0 0x7fe24019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe24019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59424/0 (socket says 172.21.15.19:59424) 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 -- 172.21.15.19:0/3890054093 learned_addr learned my addr 172.21.15.19:0/3890054093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24622f640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe240108900 0x7fe2401a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 -- 172.21.15.19:0/3890054093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2401029c0 msgr2=0x7fe24019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2401029c0 0x7fe24019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 -- 172.21.15.19:0/3890054093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe240108900 msgr2=0x7fe2401a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe240108900 0x7fe2401a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 -- 172.21.15.19:0/3890054093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2401a1d50 con 0x7fe240103c00 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.283+0000 7fe24522d640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe24019a750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe23c006010 tx=0x7fe23c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.284+0000 7fe236ffd640 1 -- 172.21.15.19:0/3890054093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c03f030 con 0x7fe240103c00 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.284+0000 7fe236ffd640 1 -- 172.21.15.19:0/3890054093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe23c031910 con 0x7fe240103c00 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.284+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2401a1fe0 con 0x7fe240103c00 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.284+0000 7fe236ffd640 1 -- 172.21.15.19:0/3890054093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c035070 con 0x7fe240103c00 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.284+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2401a2440 con 0x7fe240103c00 2024-04-25T10:52:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.285+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2401a26a0 con 0x7fe240103c00 2024-04-25T10:52:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.287+0000 7fe236ffd640 1 -- 172.21.15.19:0/3890054093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe23c042030 con 0x7fe240103c00 2024-04-25T10:52:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.288+0000 7fe236ffd640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe22c0afb70 0x7fe22c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.288+0000 7fe245a2e640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe22c0afb70 0x7fe22c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.288+0000 7fe236ffd640 1 -- 172.21.15.19:0/3890054093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe23c0ff2c0 con 0x7fe240103c00 2024-04-25T10:52:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.288+0000 7fe245a2e640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe22c0afb70 0x7fe22c0b2080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2401035c0 tx=0x7fe230005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.291+0000 7fe236ffd640 1 -- 172.21.15.19:0/3890054093 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe23c0c7c50 con 0x7fe240103c00 2024-04-25T10:52:28.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.523+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2401a2a30 con 0x7fe240103c00 2024-04-25T10:52:28.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.524+0000 7fe236ffd640 1 -- 172.21.15.19:0/3890054093 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7fe23c0cbf50 con 0x7fe240103c00 2024-04-25T10:52:28.525 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:28.525 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24533,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T10:52:28.525 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe22c0afb70 msgr2=0x7fe22c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe22c0afb70 0x7fe22c0b2080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2401035c0 tx=0x7fe230005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 msgr2=0x7fe24019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe24019a750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe23c006010 tx=0x7fe23c0042f0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 shutdown_connections 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe240108900 0x7fe2401a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe22c0afb70 0x7fe22c0b2080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe240103c00 0x7fe24019a750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 --2- 172.21.15.19:0/3890054093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2401029c0 0x7fe24019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 >> 172.21.15.19:0/3890054093 conn(0x7fe2400fdbd0 msgr2=0x7fe240109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:28.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 shutdown_connections 2024-04-25T10:52:28.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.526+0000 7fe247c92640 1 -- 172.21.15.19:0/3890054093 wait complete. 2024-04-25T10:52:28.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:52:28.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.650+0000 7f8a556b6640 1 -- 172.21.15.19:0/2134284494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50103c00 msgr2=0x7f8a50104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.650+0000 7f8a556b6640 1 --2- 172.21.15.19:0/2134284494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50103c00 0x7f8a50104010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009900 tx=0x7f8a4402a410 comp rx=0 tx=0).stop 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.650+0000 7f8a556b6640 1 -- 172.21.15.19:0/2134284494 shutdown_connections 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.650+0000 7f8a556b6640 1 --2- 172.21.15.19:0/2134284494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a50108900 0x7f8a5010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.650+0000 7f8a556b6640 1 --2- 172.21.15.19:0/2134284494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50103c00 0x7f8a50104010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.650+0000 7f8a556b6640 1 --2- 172.21.15.19:0/2134284494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a501029c0 0x7f8a501036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.650+0000 7f8a556b6640 1 -- 172.21.15.19:0/2134284494 >> 172.21.15.19:0/2134284494 conn(0x7f8a500fdbd0 msgr2=0x7f8a50100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.651+0000 7f8a556b6640 1 -- 172.21.15.19:0/2134284494 shutdown_connections 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.651+0000 7f8a556b6640 1 -- 172.21.15.19:0/2134284494 wait complete. 2024-04-25T10:52:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.651+0000 7f8a556b6640 1 Processor -- start 2024-04-25T10:52:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.651+0000 7f8a556b6640 1 -- start start 2024-04-25T10:52:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a556b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a501029c0 0x7f8a5006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a556b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a50103c00 0x7f8a5019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a556b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50108900 0x7f8a501a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a556b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5010e530 con 0x7f8a50108900 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a556b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5010e3b0 con 0x7f8a501029c0 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a556b6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5010e6b0 con 0x7f8a50103c00 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50108900 0x7f8a501a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a50103c00 0x7f8a5019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50108900 0x7f8a501a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59442/0 (socket says 172.21.15.19:59442) 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a50103c00 0x7f8a5019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:37086/0 (socket says 172.21.15.19:37086) 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 -- 172.21.15.19:0/357038389 learned_addr learned my addr 172.21.15.19:0/357038389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4effd640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a501029c0 0x7f8a5006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 -- 172.21.15.19:0/357038389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a50103c00 msgr2=0x7f8a5019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a50103c00 0x7f8a5019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 -- 172.21.15.19:0/357038389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a501029c0 msgr2=0x7f8a5006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a501029c0 0x7f8a5006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 -- 172.21.15.19:0/357038389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a501a1d30 con 0x7f8a50108900 2024-04-25T10:52:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.652+0000 7f8a4f7fe640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50108900 0x7f8a501a1510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800eaa0 tx=0x7f8a4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.653+0000 7f8a37fff640 1 -- 172.21.15.19:0/357038389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800cdb0 con 0x7f8a50108900 2024-04-25T10:52:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.653+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a501a1fc0 con 0x7f8a50108900 2024-04-25T10:52:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.653+0000 7f8a37fff640 1 -- 172.21.15.19:0/357038389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a48005020 con 0x7f8a50108900 2024-04-25T10:52:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.653+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a2480 con 0x7f8a50108900 2024-04-25T10:52:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.653+0000 7f8a37fff640 1 -- 172.21.15.19:0/357038389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480116d0 con 0x7f8a50108900 2024-04-25T10:52:28.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.654+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a50107710 con 0x7f8a50108900 2024-04-25T10:52:28.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.655+0000 7f8a37fff640 1 -- 172.21.15.19:0/357038389 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8a480051d0 con 0x7f8a50108900 2024-04-25T10:52:28.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.656+0000 7f8a37fff640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a380afaf0 0x7f8a380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:28.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.656+0000 7f8a37fff640 1 -- 172.21.15.19:0/357038389 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8a480de8c0 con 0x7f8a50108900 2024-04-25T10:52:28.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.656+0000 7f8a4effd640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a380afaf0 0x7f8a380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:28.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.657+0000 7f8a4effd640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a380afaf0 0x7f8a380b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005fd0 tx=0x7f8a4000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.662+0000 7f8a37fff640 1 -- 172.21.15.19:0/357038389 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a48016030 con 0x7f8a50108900 2024-04-25T10:52:28.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:28 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] up:boot 2024-04-25T10:52:28.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:28 smithi169 ceph-mon[33049]: fsmap 12 up:standby 2024-04-25T10:52:28.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:28 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:52:28.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:28 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3890054093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:28 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] up:boot 2024-04-25T10:52:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:28 smithi019 ceph-mon[29209]: fsmap 12 up:standby 2024-04-25T10:52:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:28 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:52:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:28 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3890054093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.876+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a500637a0 con 0x7f8a50108900 2024-04-25T10:52:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.877+0000 7f8a37fff640 1 -- 172.21.15.19:0/357038389 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18463 (secure 0 0 0) 0x7f8a480a72f0 con 0x7f8a50108900 2024-04-25T10:52:28.878 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:28.878 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":73,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:51:45.338653+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a380afaf0 msgr2=0x7f8a380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a380afaf0 0x7f8a380b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005fd0 tx=0x7f8a4000b390 comp rx=0 tx=0).stop 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50108900 msgr2=0x7f8a501a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50108900 0x7f8a501a1510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800eaa0 tx=0x7f8a4800ef60 comp rx=0 tx=0).stop 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 shutdown_connections 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a380afaf0 0x7f8a380b2000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a50108900 0x7f8a501a1510 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a50103c00 0x7f8a5019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 --2- 172.21.15.19:0/357038389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a501029c0 0x7f8a5006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 >> 172.21.15.19:0/357038389 conn(0x7f8a500fdbd0 msgr2=0x7f8a501048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 shutdown_connections 2024-04-25T10:52:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:28.879+0000 7f8a556b6640 1 -- 172.21.15.19:0/357038389 wait complete. 2024-04-25T10:52:28.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:52:28.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:52:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 -- 172.21.15.19:0/488726993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 msgr2=0x7f2464104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 --2- 172.21.15.19:0/488726993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 0x7f2464104030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2454009860 tx=0x7f245402a340 comp rx=0 tx=0).stop 2024-04-25T10:52:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 -- 172.21.15.19:0/488726993 shutdown_connections 2024-04-25T10:52:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 --2- 172.21.15.19:0/488726993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464108920 0x7f246410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 --2- 172.21.15.19:0/488726993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 0x7f2464104030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 --2- 172.21.15.19:0/488726993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24641029e0 0x7f24641036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 -- 172.21.15.19:0/488726993 >> 172.21.15.19:0/488726993 conn(0x7f24640fdbd0 msgr2=0x7f2464100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 -- 172.21.15.19:0/488726993 shutdown_connections 2024-04-25T10:52:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.015+0000 7f246b5ee640 1 -- 172.21.15.19:0/488726993 wait complete. 2024-04-25T10:52:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 Processor -- start 2024-04-25T10:52:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 -- start start 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24641029e0 0x7f2464199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 0x7f2464199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464108920 0x7f24641a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f246410e510 con 0x7f2464108920 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f246410e390 con 0x7f24641029e0 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246b5ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f246410e690 con 0x7f2464103c20 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f2469b8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464108920 0x7f24641a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f2469b8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464108920 0x7f24641a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59458/0 (socket says 172.21.15.19:59458) 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f2469b8b640 1 -- 172.21.15.19:0/2551866770 learned_addr learned my addr 172.21.15.19:0/2551866770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f246938a640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24641029e0 0x7f2464199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.016+0000 7f2468b89640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 0x7f2464199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246938a640 1 -- 172.21.15.19:0/2551866770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 msgr2=0x7f2464199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246938a640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 0x7f2464199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246938a640 1 -- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464108920 msgr2=0x7f24641a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246938a640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464108920 0x7f24641a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246938a640 1 -- 172.21.15.19:0/2551866770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24641a0f70 con 0x7f24641029e0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246938a640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24641029e0 0x7f2464199430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24641035e0 tx=0x7f246000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f245a7fc640 1 -- 172.21.15.19:0/2551866770 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460004440 con 0x7f24641029e0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24641a1260 con 0x7f24641029e0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f245a7fc640 1 -- 172.21.15.19:0/2551866770 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f246000cd70 con 0x7f24641029e0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f245a7fc640 1 -- 172.21.15.19:0/2551866770 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460019480 con 0x7f24641029e0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.017+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24641a17b0 con 0x7f24641029e0 2024-04-25T10:52:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.018+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2464107730 con 0x7f24641029e0 2024-04-25T10:52:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.020+0000 7f245a7fc640 1 -- 172.21.15.19:0/2551866770 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f246000c8c0 con 0x7f24641029e0 2024-04-25T10:52:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.021+0000 7f245a7fc640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24500afaf0 0x7f24500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.021+0000 7f2468b89640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24500afaf0 0x7f24500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.021+0000 7f245a7fc640 1 -- 172.21.15.19:0/2551866770 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f24600deb30 con 0x7f24641029e0 2024-04-25T10:52:29.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.021+0000 7f2468b89640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24500afaf0 0x7f24500b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f245402a930 tx=0x7f2454030450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.025+0000 7f245a7fc640 1 -- 172.21.15.19:0/2551866770 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2460009510 con 0x7f24641029e0 2024-04-25T10:52:29.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:28 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] up:boot 2024-04-25T10:52:29.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:28 smithi112 ceph-mon[32799]: fsmap 12 up:standby 2024-04-25T10:52:29.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:28 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:52:29.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:28 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3890054093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.242+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f246406c2a0 con 0x7f24641029e0 2024-04-25T10:52:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.243+0000 7f245a7fc640 1 -- 172.21.15.19:0/2551866770 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f24600a7b90 con 0x7f24641029e0 2024-04-25T10:52:29.244 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24500afaf0 msgr2=0x7f24500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24500afaf0 0x7f24500b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f245402a930 tx=0x7f2454030450 comp rx=0 tx=0).stop 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24641029e0 msgr2=0x7f2464199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24641029e0 0x7f2464199430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24641035e0 tx=0x7f246000ee50 comp rx=0 tx=0).stop 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 shutdown_connections 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24500afaf0 0x7f24500b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464108920 0x7f24641a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2464103c20 0x7f2464199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 --2- 172.21.15.19:0/2551866770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24641029e0 0x7f2464199430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 >> 172.21.15.19:0/2551866770 conn(0x7f24640fdbd0 msgr2=0x7f2464104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 shutdown_connections 2024-04-25T10:52:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.245+0000 7f246b5ee640 1 -- 172.21.15.19:0/2551866770 wait complete. 2024-04-25T10:52:29.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:52:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 -- 172.21.15.19:0/2041593325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a81045d0 msgr2=0x7f60a81069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 --2- 172.21.15.19:0/2041593325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a81045d0 0x7f60a81069f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6098009900 tx=0x7f609802a410 comp rx=0 tx=0).stop 2024-04-25T10:52:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 -- 172.21.15.19:0/2041593325 shutdown_connections 2024-04-25T10:52:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 --2- 172.21.15.19:0/2041593325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a8109890 0x7f60a810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 --2- 172.21.15.19:0/2041593325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a8106f30 0x7f60a8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 --2- 172.21.15.19:0/2041593325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a81045d0 0x7f60a81069f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 -- 172.21.15.19:0/2041593325 >> 172.21.15.19:0/2041593325 conn(0x7f60a80fdbd0 msgr2=0x7f60a8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 -- 172.21.15.19:0/2041593325 shutdown_connections 2024-04-25T10:52:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.370+0000 7f60ada05640 1 -- 172.21.15.19:0/2041593325 wait complete. 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.371+0000 7f60ada05640 1 Processor -- start 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.371+0000 7f60ada05640 1 -- start start 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60ada05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a81045d0 0x7f60a8103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60ada05640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8106f30 0x7f60a81020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60ada05640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a8109890 0x7f60a8102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60ada05640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810f5e0 con 0x7f60a81045d0 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60ada05640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810f460 con 0x7f60a8106f30 2024-04-25T10:52:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60ada05640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810f760 con 0x7f60a8109890 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a81045d0 0x7f60a8103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a81045d0 0x7f60a8103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59478/0 (socket says 172.21.15.19:59478) 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 -- 172.21.15.19:0/340191364 learned_addr learned my addr 172.21.15.19:0/340191364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a67fc640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8106f30 0x7f60a81020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 -- 172.21.15.19:0/340191364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a8109890 msgr2=0x7f60a8102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a8109890 0x7f60a8102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 -- 172.21.15.19:0/340191364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8106f30 msgr2=0x7f60a81020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8106f30 0x7f60a81020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 -- 172.21.15.19:0/340191364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a8102d30 con 0x7f60a81045d0 2024-04-25T10:52:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a67fc640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8106f30 0x7f60a81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60a6ffd640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a81045d0 0x7f60a8103a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6098006010 tx=0x7f6098004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60aca03640 1 -- 172.21.15.19:0/340191364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609803f030 con 0x7f60a81045d0 2024-04-25T10:52:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.372+0000 7f60aca03640 1 -- 172.21.15.19:0/340191364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6098031910 con 0x7f60a81045d0 2024-04-25T10:52:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.373+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a81a42d0 con 0x7f60a81045d0 2024-04-25T10:52:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.373+0000 7f60aca03640 1 -- 172.21.15.19:0/340191364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098035070 con 0x7f60a81045d0 2024-04-25T10:52:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.373+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a81a4730 con 0x7f60a81045d0 2024-04-25T10:52:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.374+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a80ff6f0 con 0x7f60a81045d0 2024-04-25T10:52:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.375+0000 7f60aca03640 1 -- 172.21.15.19:0/340191364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6098042030 con 0x7f60a81045d0 2024-04-25T10:52:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.376+0000 7f60aca03640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f60900afb70 0x7f60900b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.376+0000 7f60a67fc640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f60900afb70 0x7f60900b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.376+0000 7f60aca03640 1 -- 172.21.15.19:0/340191364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f60980fef00 con 0x7f60a81045d0 2024-04-25T10:52:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.376+0000 7f60a67fc640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f60900afb70 0x7f60900b2080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60a81033c0 tx=0x7f609c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:29.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.380+0000 7f60aca03640 1 -- 172.21.15.19:0/340191364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60980c7f80 con 0x7f60a81045d0 2024-04-25T10:52:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.570+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f60a802ddc0 con 0x7f60a81045d0 2024-04-25T10:52:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.570+0000 7f60aca03640 1 -- 172.21.15.19:0/340191364 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f60980c7880 con 0x7f60a81045d0 2024-04-25T10:52:29.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:29.571 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:52:29.572 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:52:29.572 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:52:29.572 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:52:29.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:52:29.572 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:52:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f60900afb70 msgr2=0x7f60900b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f60900afb70 0x7f60900b2080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60a81033c0 tx=0x7f609c005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a81045d0 msgr2=0x7f60a8103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a81045d0 0x7f60a8103a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6098006010 tx=0x7f6098004270 comp rx=0 tx=0).stop 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 shutdown_connections 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a8109890 0x7f60a8102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8106f30 0x7f60a81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f60900afb70 0x7f60900b2080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 --2- 172.21.15.19:0/340191364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a81045d0 0x7f60a8103a20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.572+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 >> 172.21.15.19:0/340191364 conn(0x7f60a80fdbd0 msgr2=0x7f60a8105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.573+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 shutdown_connections 2024-04-25T10:52:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.573+0000 7f60ada05640 1 -- 172.21.15.19:0/340191364 wait complete. 2024-04-25T10:52:29.585 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:52:29.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:52:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.702+0000 7fa22b692640 1 -- 172.21.15.19:0/415760460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 msgr2=0x7fa22410adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.702+0000 7fa22b692640 1 --2- 172.21.15.19:0/415760460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa22410adb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009ca0 tx=0x7fa21c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.702+0000 7fa22b692640 1 -- 172.21.15.19:0/415760460 shutdown_connections 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.702+0000 7fa22b692640 1 --2- 172.21.15.19:0/415760460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa22410adb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.703+0000 7fa22b692640 1 --2- 172.21.15.19:0/415760460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224103c90 0x7fa2241040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.703+0000 7fa22b692640 1 --2- 172.21.15.19:0/415760460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa224102a50 0x7fa224103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.703+0000 7fa22b692640 1 -- 172.21.15.19:0/415760460 >> 172.21.15.19:0/415760460 conn(0x7fa2240fdc40 msgr2=0x7fa2241000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.703+0000 7fa22b692640 1 -- 172.21.15.19:0/415760460 shutdown_connections 2024-04-25T10:52:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.703+0000 7fa22b692640 1 -- 172.21.15.19:0/415760460 wait complete. 2024-04-25T10:52:29.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.703+0000 7fa22b692640 1 Processor -- start 2024-04-25T10:52:29.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.703+0000 7fa22b692640 1 -- start start 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22b692640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa224102a50 0x7fa224075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22b692640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224103c90 0x7fa22419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22b692640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa2241a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22b692640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa22410e510 con 0x7fa224108990 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22b692640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa22410e390 con 0x7fa224102a50 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22b692640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa22410e690 con 0x7fa224103c90 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa2241a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa2241a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59494/0 (socket says 172.21.15.19:59494) 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 -- 172.21.15.19:0/1719325436 learned_addr learned my addr 172.21.15.19:0/1719325436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa228c2d640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224103c90 0x7fa22419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22942e640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa224102a50 0x7fa224075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 -- 172.21.15.19:0/1719325436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224103c90 msgr2=0x7fa22419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224103c90 0x7fa22419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 -- 172.21.15.19:0/1719325436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa224102a50 msgr2=0x7fa224075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa224102a50 0x7fa224075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 -- 172.21.15.19:0/1719325436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2241a1cd0 con 0x7fa224108990 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa228c2d640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224103c90 0x7fa22419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa22942e640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa224102a50 0x7fa224075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa229c2f640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa2241a14b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa21c004af0 tx=0x7fa21c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.704+0000 7fa21a7fc640 1 -- 172.21.15.19:0/1719325436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c003b70 con 0x7fa224108990 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.705+0000 7fa21a7fc640 1 -- 172.21.15.19:0/1719325436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa21c003d10 con 0x7fa224108990 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.705+0000 7fa21a7fc640 1 -- 172.21.15.19:0/1719325436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c02fb20 con 0x7fa224108990 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.705+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2241a1f60 con 0x7fa224108990 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.705+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2241a2370 con 0x7fa224108990 2024-04-25T10:52:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.706+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22406a190 con 0x7fa224108990 2024-04-25T10:52:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.707+0000 7fa21a7fc640 1 -- 172.21.15.19:0/1719325436 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa21c004580 con 0x7fa224108990 2024-04-25T10:52:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.708+0000 7fa21a7fc640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa2100afaf0 0x7fa2100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.708+0000 7fa22942e640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa2100afaf0 0x7fa2100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.708+0000 7fa21a7fc640 1 -- 172.21.15.19:0/1719325436 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fa21c0ffea0 con 0x7fa224108990 2024-04-25T10:52:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.708+0000 7fa22942e640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa2100afaf0 0x7fa2100b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa224103650 tx=0x7fa214005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:29.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.711+0000 7fa21a7fc640 1 -- 172.21.15.19:0/1719325436 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa22406a190 con 0x7fa224108990 2024-04-25T10:52:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/357038389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:52:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2551866770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:52:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:52:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/340191364' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T10:52:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:29.906+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa22402dc10 con 0x7fa224108990 2024-04-25T10:52:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/357038389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2551866770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:52:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/340191364' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T10:52:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/357038389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2551866770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:52:30.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:29 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/340191364' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T10:52:30.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.676+0000 7fa21a7fc640 1 -- 172.21.15.19:0/1719325436 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7fa21c0c8f20 con 0x7fa224108990 2024-04-25T10:52:30.678 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:52:30.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa2100afaf0 msgr2=0x7fa2100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:30.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa2100afaf0 0x7fa2100b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa224103650 tx=0x7fa214005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:30.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 msgr2=0x7fa2241a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:30.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa2241a14b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa21c004af0 tx=0x7fa21c031700 comp rx=0 tx=0).stop 2024-04-25T10:52:30.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 shutdown_connections 2024-04-25T10:52:30.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa2100afaf0 0x7fa2100b2000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa224108990 0x7fa2241a14b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224103c90 0x7fa22419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 --2- 172.21.15.19:0/1719325436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa224102a50 0x7fa224075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 >> 172.21.15.19:0/1719325436 conn(0x7fa2240fdc40 msgr2=0x7fa224104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 shutdown_connections 2024-04-25T10:52:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.679+0000 7fa22b692640 1 -- 172.21.15.19:0/1719325436 wait complete. 2024-04-25T10:52:30.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 -- 172.21.15.19:0/127414833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec1045b0 msgr2=0x7faaec1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 --2- 172.21.15.19:0/127414833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec1045b0 0x7faaec1069d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faae8009900 tx=0x7faae802a410 comp rx=0 tx=0).stop 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 -- 172.21.15.19:0/127414833 shutdown_connections 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 --2- 172.21.15.19:0/127414833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaec109870 0x7faaec10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 --2- 172.21.15.19:0/127414833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec106f10 0x7faaec109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 --2- 172.21.15.19:0/127414833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec1045b0 0x7faaec1069d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 -- 172.21.15.19:0/127414833 >> 172.21.15.19:0/127414833 conn(0x7faaec0fdbd0 msgr2=0x7faaec100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 -- 172.21.15.19:0/127414833 shutdown_connections 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.821+0000 7faaf4772640 1 -- 172.21.15.19:0/127414833 wait complete. 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 Processor -- start 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 -- start start 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaec1045b0 0x7faaec19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec106f10 0x7faaec19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec109870 0x7faaec1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faaec10f5c0 con 0x7faaec106f10 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaec10f440 con 0x7faaec109870 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.822+0000 7faaf4772640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faaec10f740 con 0x7faaec1045b0 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf1d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec106f10 0x7faaec19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf1d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec106f10 0x7faaec19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59514/0 (socket says 172.21.15.19:59514) 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf1d0d640 1 -- 172.21.15.19:0/875656697 learned_addr learned my addr 172.21.15.19:0/875656697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf2d0f640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec109870 0x7faaec1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf250e640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaec1045b0 0x7faaec19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf2d0f640 1 -- 172.21.15.19:0/875656697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaec1045b0 msgr2=0x7faaec19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf2d0f640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaec1045b0 0x7faaec19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf2d0f640 1 -- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec106f10 msgr2=0x7faaec19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf2d0f640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec106f10 0x7faaec19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf2d0f640 1 -- 172.21.15.19:0/875656697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec1a1d60 con 0x7faaec109870 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf1d0d640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec106f10 0x7faaec19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf250e640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaec1045b0 0x7faaec19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf2d0f640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec109870 0x7faaec1a16f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faae4005fd0 tx=0x7faae400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faae37fe640 1 -- 172.21.15.19:0/875656697 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400cdf0 con 0x7faaec109870 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faae37fe640 1 -- 172.21.15.19:0/875656697 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae4005020 con 0x7faaec109870 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.823+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec1a1ff0 con 0x7faaec109870 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.824+0000 7faae37fe640 1 -- 172.21.15.19:0/875656697 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4010690 con 0x7faaec109870 2024-04-25T10:52:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.824+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1a24b0 con 0x7faaec109870 2024-04-25T10:52:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.825+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec0ff6f0 con 0x7faaec109870 2024-04-25T10:52:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.826+0000 7faae37fe640 1 -- 172.21.15.19:0/875656697 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7faae4005460 con 0x7faaec109870 2024-04-25T10:52:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.827+0000 7faae37fe640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faad80afaf0 0x7faad80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.827+0000 7faaf250e640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faad80afaf0 0x7faad80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.827+0000 7faae37fe640 1 -- 172.21.15.19:0/875656697 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7faae40def40 con 0x7faaec109870 2024-04-25T10:52:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.827+0000 7faaf250e640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faad80afaf0 0x7faad80b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faae8009230 tx=0x7faae8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:30.833+0000 7faae37fe640 1 -- 172.21.15.19:0/875656697 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faae40a7e30 con 0x7faaec109870 2024-04-25T10:52:31.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:30 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1719325436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T10:52:31.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:31.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:30 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1719325436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T10:52:31.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.060+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faaec02db50 con 0x7faaec109870 2024-04-25T10:52:31.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:30 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1719325436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T10:52:31.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:31.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.684+0000 7faae37fe640 1 -- 172.21.15.19:0/875656697 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7faae40a7730 con 0x7faaec109870 2024-04-25T10:52:31.686 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:52:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faad80afaf0 msgr2=0x7faad80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faad80afaf0 0x7faad80b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faae8009230 tx=0x7faae8031400 comp rx=0 tx=0).stop 2024-04-25T10:52:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec109870 msgr2=0x7faaec1a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec109870 0x7faaec1a16f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faae4005fd0 tx=0x7faae400ef10 comp rx=0 tx=0).stop 2024-04-25T10:52:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 shutdown_connections 2024-04-25T10:52:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec109870 0x7faaec1a16f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faad80afaf0 0x7faad80b2000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaec106f10 0x7faaec19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 --2- 172.21.15.19:0/875656697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaec1045b0 0x7faaec19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 >> 172.21.15.19:0/875656697 conn(0x7faaec0fdbd0 msgr2=0x7faaec1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:31.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 shutdown_connections 2024-04-25T10:52:31.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.689+0000 7faaf4772640 1 -- 172.21.15.19:0/875656697 wait complete. 2024-04-25T10:52:31.713 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:52:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/4068804426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8109a30 msgr2=0x7f1bb810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/4068804426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8109a30 0x7f1bb810be50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009ca0 tx=0x7f1bb002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/4068804426 shutdown_connections 2024-04-25T10:52:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/4068804426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8109a30 0x7f1bb810be50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/4068804426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb8104d30 0x7f1bb8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/4068804426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb8103af0 0x7f1bb81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/4068804426 >> 172.21.15.19:0/4068804426 conn(0x7f1bb80fece0 msgr2=0x7f1bb8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/4068804426 shutdown_connections 2024-04-25T10:52:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.831+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/4068804426 wait complete. 2024-04-25T10:52:31.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 Processor -- start 2024-04-25T10:52:31.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 -- start start 2024-04-25T10:52:31.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8103af0 0x7f1bb8077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:31.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb8104d30 0x7f1bb8076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:31.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb8109a30 0x7f1bb8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810f620 con 0x7f1bb8103af0 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810f4a0 con 0x7f1bb8104d30 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.832+0000 7f1bbc9c5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810f7a0 con 0x7f1bb8109a30 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8103af0 0x7f1bb8077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8103af0 0x7f1bb8077910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59520/0 (socket says 172.21.15.19:59520) 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 -- 172.21.15.19:0/454116893 learned_addr learned my addr 172.21.15.19:0/454116893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb6d9d640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb8109a30 0x7f1bb8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb5d9b640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb8104d30 0x7f1bb8076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 -- 172.21.15.19:0/454116893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb8109a30 msgr2=0x7f1bb8076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb8109a30 0x7f1bb8076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 -- 172.21.15.19:0/454116893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb8104d30 msgr2=0x7f1bb8076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb8104d30 0x7f1bb8076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 -- 172.21.15.19:0/454116893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb8076bb0 con 0x7f1bb8103af0 2024-04-25T10:52:31.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb6d9d640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb8109a30 0x7f1bb8076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bb659c640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8103af0 0x7f1bb8077910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c970 tx=0x7f1ba800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1ba77fe640 1 -- 172.21.15.19:0/454116893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8007c70 con 0x7f1bb8103af0 2024-04-25T10:52:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1ba77fe640 1 -- 172.21.15.19:0/454116893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ba8007e10 con 0x7f1bb8103af0 2024-04-25T10:52:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb81a42e0 con 0x7f1bb8103af0 2024-04-25T10:52:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1ba77fe640 1 -- 172.21.15.19:0/454116893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80187b0 con 0x7f1bb8103af0 2024-04-25T10:52:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.833+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb81a46e0 con 0x7f1bb8103af0 2024-04-25T10:52:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.834+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb806a160 con 0x7f1bb8103af0 2024-04-25T10:52:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.836+0000 7f1ba77fe640 1 -- 172.21.15.19:0/454116893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1ba800fab0 con 0x7f1bb8103af0 2024-04-25T10:52:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.836+0000 7f1ba77fe640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1ba00afaf0 0x7f1ba00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.837+0000 7f1bb5d9b640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1ba00afaf0 0x7f1ba00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.837+0000 7f1ba77fe640 1 -- 172.21.15.19:0/454116893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7879+0+0 (secure 0 0 0) 0x7f1ba80deb20 con 0x7f1bb8103af0 2024-04-25T10:52:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.837+0000 7f1bb5d9b640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1ba00afaf0 0x7f1ba00b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005fd0 tx=0x7f1bac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:31.842+0000 7f1ba77fe640 1 -- 172.21.15.19:0/454116893 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ba8014030 con 0x7f1bb8103af0 2024-04-25T10:52:32.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:32.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1719325436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T10:52:32.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T10:52:32.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/875656697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:52:32.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:31 smithi169 ceph-mon[33049]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1719325436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:52:32.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/875656697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:52:32.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:31 smithi112 ceph-mon[32799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:52:32.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.069+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1bb8108840 con 0x7f1bb8103af0 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1719325436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:52:32.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/875656697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:52:32.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:31 smithi019 ceph-mon[29209]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:52:32.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.706+0000 7f1ba77fe640 1 -- 172.21.15.19:0/454116893 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f1ba80a78b0 con 0x7f1bb8103af0 2024-04-25T10:52:32.707 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:52:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1ba00afaf0 msgr2=0x7f1ba00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1ba00afaf0 0x7f1ba00b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005fd0 tx=0x7f1bac005850 comp rx=0 tx=0).stop 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8103af0 msgr2=0x7f1bb8077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8103af0 0x7f1bb8077910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c970 tx=0x7f1ba800ce30 comp rx=0 tx=0).stop 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 shutdown_connections 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb8109a30 0x7f1bb8076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb8104d30 0x7f1bb8076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1ba00afaf0 0x7f1ba00b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 --2- 172.21.15.19:0/454116893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb8103af0 0x7f1bb8077910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 >> 172.21.15.19:0/454116893 conn(0x7f1bb80fece0 msgr2=0x7f1bb8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 shutdown_connections 2024-04-25T10:52:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.709+0000 7f1bbc9c5640 1 -- 172.21.15.19:0/454116893 wait complete. 2024-04-25T10:52:32.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:52:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.844+0000 7fcea4434640 1 -- 172.21.15.19:0/2013160955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c1029e0 msgr2=0x7fce9c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.844+0000 7fcea4434640 1 --2- 172.21.15.19:0/2013160955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c1029e0 0x7fce9c1036e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fce98009900 tx=0x7fce9802a410 comp rx=0 tx=0).stop 2024-04-25T10:52:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 -- 172.21.15.19:0/2013160955 shutdown_connections 2024-04-25T10:52:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 --2- 172.21.15.19:0/2013160955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce9c108920 0x7fce9c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 --2- 172.21.15.19:0/2013160955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce9c103c20 0x7fce9c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 --2- 172.21.15.19:0/2013160955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c1029e0 0x7fce9c1036e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 -- 172.21.15.19:0/2013160955 >> 172.21.15.19:0/2013160955 conn(0x7fce9c0fdbd0 msgr2=0x7fce9c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 -- 172.21.15.19:0/2013160955 shutdown_connections 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 -- 172.21.15.19:0/2013160955 wait complete. 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 Processor -- start 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 -- start start 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce9c1029e0 0x7fce9c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.845+0000 7fcea4434640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c103c20 0x7fce9c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea4434640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce9c108920 0x7fce9c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea4434640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c10e5e0 con 0x7fce9c103c20 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea4434640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c10e460 con 0x7fce9c1029e0 2024-04-25T10:52:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea4434640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c10e760 con 0x7fce9c108920 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c103c20 0x7fce9c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c103c20 0x7fce9c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41128/0 (socket says 172.21.15.19:41128) 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 -- 172.21.15.19:0/1645874034 learned_addr learned my addr 172.21.15.19:0/1645874034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea21d0640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce9c1029e0 0x7fce9c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea29d1640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce9c108920 0x7fce9c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 -- 172.21.15.19:0/1645874034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce9c108920 msgr2=0x7fce9c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce9c108920 0x7fce9c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 -- 172.21.15.19:0/1645874034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce9c1029e0 msgr2=0x7fce9c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce9c1029e0 0x7fce9c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 -- 172.21.15.19:0/1645874034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c1a1f90 con 0x7fce9c103c20 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea21d0640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce9c1029e0 0x7fce9c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea29d1640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce9c108920 0x7fce9c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:52:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea19cf640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c103c20 0x7fce9c19aa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00b4a0 tx=0x7fce8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fce937fe640 1 -- 172.21.15.19:0/1645874034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c014070 con 0x7fce9c103c20 2024-04-25T10:52:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fce937fe640 1 -- 172.21.15.19:0/1645874034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce8c010ac0 con 0x7fce9c103c20 2024-04-25T10:52:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c1a2280 con 0x7fce9c103c20 2024-04-25T10:52:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fce937fe640 1 -- 172.21.15.19:0/1645874034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c017c30 con 0x7fce9c103c20 2024-04-25T10:52:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.846+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c1a28a0 con 0x7fce9c103c20 2024-04-25T10:52:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.847+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c1a2b60 con 0x7fce9c103c20 2024-04-25T10:52:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.849+0000 7fce937fe640 1 -- 172.21.15.19:0/1645874034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fce8c017070 con 0x7fce9c103c20 2024-04-25T10:52:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.849+0000 7fce937fe640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce880afb70 0x7fce880b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.850+0000 7fcea21d0640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce880afb70 0x7fce880b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.850+0000 7fce937fe640 1 -- 172.21.15.19:0/1645874034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fce8c0de1e0 con 0x7fce9c103c20 2024-04-25T10:52:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.850+0000 7fcea21d0640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce880afb70 0x7fce880b2080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce98009230 tx=0x7fce98000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:32.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:32.853+0000 7fce937fe640 1 -- 172.21.15.19:0/1645874034 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce8c0a6f10 con 0x7fce9c103c20 2024-04-25T10:52:32.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:32 smithi019 ceph-mon[29209]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T10:52:32.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:32 smithi019 ceph-mon[29209]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T10:52:32.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:32 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/454116893' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T10:52:32.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:32 smithi019 ceph-mon[29209]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T10:52:32.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:32 smithi019 ceph-mon[29209]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T10:52:33.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:32 smithi169 ceph-mon[33049]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T10:52:33.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:32 smithi169 ceph-mon[33049]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T10:52:33.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:32 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/454116893' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T10:52:33.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:32 smithi169 ceph-mon[33049]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T10:52:33.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:32 smithi169 ceph-mon[33049]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T10:52:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.056+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce9c1a2d30 con 0x7fce9c103c20 2024-04-25T10:52:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.057+0000 7fce937fe640 1 -- 172.21.15.19:0/1645874034 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22239 (secure 0 0 0) 0x7fce8c0a6810 con 0x7fce9c103c20 2024-04-25T10:52:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:33.059 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":76,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:52:32.071090+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:52:33.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:32 smithi112 ceph-mon[32799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T10:52:33.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:32 smithi112 ceph-mon[32799]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T10:52:33.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:32 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/454116893' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T10:52:33.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:32 smithi112 ceph-mon[32799]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T10:52:33.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:32 smithi112 ceph-mon[32799]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T10:52:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce880afb70 msgr2=0x7fce880b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce880afb70 0x7fce880b2080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce98009230 tx=0x7fce98000bc0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c103c20 msgr2=0x7fce9c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c103c20 0x7fce9c19aa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00b4a0 tx=0x7fce8c00b960 comp rx=0 tx=0).stop 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 shutdown_connections 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce9c108920 0x7fce9c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fce880afb70 0x7fce880b2080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce9c103c20 0x7fce9c19aa00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 --2- 172.21.15.19:0/1645874034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce9c1029e0 0x7fce9c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.059+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 >> 172.21.15.19:0/1645874034 conn(0x7fce9c0fdbd0 msgr2=0x7fce9c109f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.060+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 shutdown_connections 2024-04-25T10:52:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.060+0000 7fcea4434640 1 -- 172.21.15.19:0/1645874034 wait complete. 2024-04-25T10:52:33.072 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 -- 172.21.15.19:0/3668893894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541033a0 msgr2=0x7f5e541037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 --2- 172.21.15.19:0/3668893894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541033a0 0x7f5e541037b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009930 tx=0x7f5e4802a420 comp rx=0 tx=0).stop 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 -- 172.21.15.19:0/3668893894 shutdown_connections 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 --2- 172.21.15.19:0/3668893894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e541080a0 0x7f5e5410a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 --2- 172.21.15.19:0/3668893894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541033a0 0x7f5e541037b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 --2- 172.21.15.19:0/3668893894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e54102360 0x7f5e54102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 -- 172.21.15.19:0/3668893894 >> 172.21.15.19:0/3668893894 conn(0x7f5e540fcef0 msgr2=0x7f5e540ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 -- 172.21.15.19:0/3668893894 shutdown_connections 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 -- 172.21.15.19:0/3668893894 wait complete. 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.183+0000 7f5e595c5640 1 Processor -- start 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e595c5640 1 -- start start 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e595c5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e54102360 0x7f5e5419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e595c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e541033a0 0x7f5e5419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e595c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541080a0 0x7f5e541a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e595c5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d3e0 con 0x7f5e541080a0 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e595c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d260 con 0x7f5e541033a0 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e595c5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d560 con 0x7f5e54102360 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541080a0 0x7f5e541a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541080a0 0x7f5e541a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41150/0 (socket says 172.21.15.19:41150) 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 -- 172.21.15.19:0/2784825815 learned_addr learned my addr 172.21.15.19:0/2784825815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e527fc640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e541033a0 0x7f5e5419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e52ffd640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e54102360 0x7f5e5419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 -- 172.21.15.19:0/2784825815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e54102360 msgr2=0x7f5e5419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e54102360 0x7f5e5419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 -- 172.21.15.19:0/2784825815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e541033a0 msgr2=0x7f5e5419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e541033a0 0x7f5e5419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 -- 172.21.15.19:0/2784825815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e541a1da0 con 0x7f5e541080a0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.184+0000 7f5e537fe640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541080a0 0x7f5e541a1730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00de10 tx=0x7f5e4c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.185+0000 7f5e3bfff640 1 -- 172.21.15.19:0/2784825815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c003b70 con 0x7f5e541080a0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.185+0000 7f5e3bfff640 1 -- 172.21.15.19:0/2784825815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e4c003d10 con 0x7f5e541080a0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.185+0000 7f5e3bfff640 1 -- 172.21.15.19:0/2784825815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00ea40 con 0x7f5e541080a0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.185+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e541a2090 con 0x7f5e541080a0 2024-04-25T10:52:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.185+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e541a2590 con 0x7f5e541080a0 2024-04-25T10:52:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.186+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e54106eb0 con 0x7f5e541080a0 2024-04-25T10:52:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.187+0000 7f5e3bfff640 1 -- 172.21.15.19:0/2784825815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f5e4c00ebe0 con 0x7f5e541080a0 2024-04-25T10:52:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.188+0000 7f5e3bfff640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5e3c0afaf0 0x7f5e3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.188+0000 7f5e52ffd640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5e3c0afaf0 0x7f5e3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.188+0000 7f5e3bfff640 1 -- 172.21.15.19:0/2784825815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f5e4c0defe0 con 0x7f5e541080a0 2024-04-25T10:52:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.189+0000 7f5e52ffd640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5e3c0afaf0 0x7f5e3c0b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e54069730 tx=0x7f5e4400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.192+0000 7f5e3bfff640 1 -- 172.21.15.19:0/2784825815 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e4c0a7660 con 0x7f5e541080a0 2024-04-25T10:52:33.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.381+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e540637a0 con 0x7f5e541080a0 2024-04-25T10:52:33.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.382+0000 7f5e3bfff640 1 -- 172.21.15.19:0/2784825815 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22239 (secure 0 0 0) 0x7f5e4c0ab960 con 0x7f5e541080a0 2024-04-25T10:52:33.383 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:33.384 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":76,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:52:32.071090+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:52:33.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5e3c0afaf0 msgr2=0x7f5e3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5e3c0afaf0 0x7f5e3c0b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e54069730 tx=0x7f5e4400b390 comp rx=0 tx=0).stop 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541080a0 msgr2=0x7f5e541a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541080a0 0x7f5e541a1730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00de10 tx=0x7f5e4c00b730 comp rx=0 tx=0).stop 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 shutdown_connections 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5e3c0afaf0 0x7f5e3c0b2000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e541080a0 0x7f5e541a1730 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e541033a0 0x7f5e5419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 --2- 172.21.15.19:0/2784825815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e54102360 0x7f5e5419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.384+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 >> 172.21.15.19:0/2784825815 conn(0x7f5e540fcef0 msgr2=0x7f5e54104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.385+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 shutdown_connections 2024-04-25T10:52:33.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.385+0000 7f5e595c5640 1 -- 172.21.15.19:0/2784825815 wait complete. 2024-04-25T10:52:33.397 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:52:33.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 -- 172.21.15.19:0/2665356385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c104d30 msgr2=0x7fad7c105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 --2- 172.21.15.19:0/2665356385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c104d30 0x7fad7c105140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009900 tx=0x7fad6c02a410 comp rx=0 tx=0).stop 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 -- 172.21.15.19:0/2665356385 shutdown_connections 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 --2- 172.21.15.19:0/2665356385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad7c109a30 0x7fad7c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 --2- 172.21.15.19:0/2665356385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c104d30 0x7fad7c105140 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 --2- 172.21.15.19:0/2665356385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad7c103af0 0x7fad7c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 -- 172.21.15.19:0/2665356385 >> 172.21.15.19:0/2665356385 conn(0x7fad7c0fece0 msgr2=0x7fad7c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 -- 172.21.15.19:0/2665356385 shutdown_connections 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 -- 172.21.15.19:0/2665356385 wait complete. 2024-04-25T10:52:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 Processor -- start 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.512+0000 7fad83a67640 1 -- start start 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad83a67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c103af0 0x7fad7c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad83a67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad7c104d30 0x7fad7c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad83a67640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad7c109a30 0x7fad7c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad83a67640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10f640 con 0x7fad7c103af0 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad83a67640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10f4c0 con 0x7fad7c104d30 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad83a67640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10f7c0 con 0x7fad7c109a30 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c103af0 0x7fad7c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c103af0 0x7fad7c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41174/0 (socket says 172.21.15.19:41174) 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 -- 172.21.15.19:0/1590975050 learned_addr learned my addr 172.21.15.19:0/1590975050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81002640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad7c104d30 0x7fad7c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad82004640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad7c109a30 0x7fad7c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 -- 172.21.15.19:0/1590975050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad7c109a30 msgr2=0x7fad7c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad7c109a30 0x7fad7c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 -- 172.21.15.19:0/1590975050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad7c104d30 msgr2=0x7fad7c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad7c104d30 0x7fad7c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 -- 172.21.15.19:0/1590975050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c1a1d50 con 0x7fad7c103af0 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad82004640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad7c109a30 0x7fad7c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81002640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad7c104d30 0x7fad7c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad81803640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c103af0 0x7fad7c19a210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9f0 tx=0x7fad7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad72ffd640 1 -- 172.21.15.19:0/1590975050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7400ccc0 con 0x7fad7c103af0 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad72ffd640 1 -- 172.21.15.19:0/1590975050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad740035f0 con 0x7fad7c103af0 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c1a1fe0 con 0x7fad7c103af0 2024-04-25T10:52:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.513+0000 7fad72ffd640 1 -- 172.21.15.19:0/1590975050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7400ce60 con 0x7fad7c103af0 2024-04-25T10:52:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.514+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c1a24f0 con 0x7fad7c103af0 2024-04-25T10:52:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.515+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7c108840 con 0x7fad7c103af0 2024-04-25T10:52:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.516+0000 7fad72ffd640 1 -- 172.21.15.19:0/1590975050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fad74003aa0 con 0x7fad7c103af0 2024-04-25T10:52:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.516+0000 7fad72ffd640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fad680afb70 0x7fad680b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.517+0000 7fad81002640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fad680afb70 0x7fad680b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.517+0000 7fad72ffd640 1 -- 172.21.15.19:0/1590975050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fad740dee60 con 0x7fad7c103af0 2024-04-25T10:52:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.517+0000 7fad81002640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fad680afb70 0x7fad680b2080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fad6c006010 tx=0x7fad6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:33.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.521+0000 7fad72ffd640 1 -- 172.21.15.19:0/1590975050 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad740a7b90 con 0x7fad7c103af0 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: pgmap v276: 97 pgs: 25 creating+peering, 3 active+clean, 69 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/454116893' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1645874034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2784825815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:33.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:33 smithi019 ceph-mon[29209]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T10:52:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:33.752+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fad7c02db50 con 0x7fad7c103af0 2024-04-25T10:52:34.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: pgmap v276: 97 pgs: 25 creating+peering, 3 active+clean, 69 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:34.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T10:52:34.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/454116893' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T10:52:34.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T10:52:34.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T10:52:34.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1645874034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:34.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2784825815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:34.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:33 smithi169 ceph-mon[33049]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: pgmap v276: 97 pgs: 25 creating+peering, 3 active+clean, 69 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/454116893' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1645874034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2784825815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:34.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:33 smithi112 ceph-mon[32799]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T10:52:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.722+0000 7fad72ffd640 1 -- 172.21.15.19:0/1590975050 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fad740a7490 con 0x7fad7c103af0 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fad680afb70 msgr2=0x7fad680b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fad680afb70 0x7fad680b2080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fad6c006010 tx=0x7fad6c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c103af0 msgr2=0x7fad7c19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c103af0 0x7fad7c19a210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9f0 tx=0x7fad7400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 shutdown_connections 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad7c109a30 0x7fad7c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad7c104d30 0x7fad7c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fad680afb70 0x7fad680b2080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 --2- 172.21.15.19:0/1590975050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c103af0 0x7fad7c19a210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 >> 172.21.15.19:0/1590975050 conn(0x7fad7c0fece0 msgr2=0x7fad7c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 shutdown_connections 2024-04-25T10:52:34.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.726+0000 7fad83a67640 1 -- 172.21.15.19:0/1590975050 wait complete. 2024-04-25T10:52:34.741 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T10:52:34.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:52:34.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 -- 172.21.15.19:0/3617571980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 msgr2=0x7fcb80104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3617571980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb80104030 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009900 tx=0x7fcb7002a410 comp rx=0 tx=0).stop 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 -- 172.21.15.19:0/3617571980 shutdown_connections 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3617571980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb80108920 0x7fcb8010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3617571980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb80104030 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3617571980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb801029e0 0x7fcb801036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 -- 172.21.15.19:0/3617571980 >> 172.21.15.19:0/3617571980 conn(0x7fcb800fdbd0 msgr2=0x7fcb80100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 -- 172.21.15.19:0/3617571980 shutdown_connections 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.862+0000 7fcb8817a640 1 -- 172.21.15.19:0/3617571980 wait complete. 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 Processor -- start 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 -- start start 2024-04-25T10:52:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb801029e0 0x7fcb8019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb8019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80108920 0x7fcb801a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e560 con 0x7fcb80103c20 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e3e0 con 0x7fcb80108920 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb8817a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e6e0 con 0x7fcb801029e0 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb8019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb8019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41192/0 (socket says 172.21.15.19:41192) 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 -- 172.21.15.19:0/3053591067 learned_addr learned my addr 172.21.15.19:0/3053591067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb86717640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80108920 0x7fcb801a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 -- 172.21.15.19:0/3053591067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb801029e0 msgr2=0x7fcb8019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85f16640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb801029e0 0x7fcb8019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb801029e0 0x7fcb8019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 -- 172.21.15.19:0/3053591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80108920 msgr2=0x7fcb801a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80108920 0x7fcb801a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.863+0000 7fcb85715640 1 -- 172.21.15.19:0/3053591067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb801a20c0 con 0x7fcb80103c20 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.864+0000 7fcb85715640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb8019aac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006010 tx=0x7fcb70004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.864+0000 7fcb76ffd640 1 -- 172.21.15.19:0/3053591067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7003f030 con 0x7fcb80103c20 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.864+0000 7fcb76ffd640 1 -- 172.21.15.19:0/3053591067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb70003cb0 con 0x7fcb80103c20 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.864+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb801a2350 con 0x7fcb80103c20 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.864+0000 7fcb76ffd640 1 -- 172.21.15.19:0/3053591067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70035070 con 0x7fcb80103c20 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.864+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb801a2840 con 0x7fcb80103c20 2024-04-25T10:52:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.865+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8006c2a0 con 0x7fcb80103c20 2024-04-25T10:52:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.866+0000 7fcb76ffd640 1 -- 172.21.15.19:0/3053591067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fcb70042030 con 0x7fcb80103c20 2024-04-25T10:52:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.867+0000 7fcb76ffd640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.867+0000 7fcb76ffd640 1 -- 172.21.15.19:0/3053591067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fcb700ff860 con 0x7fcb80103c20 2024-04-25T10:52:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.867+0000 7fcb85f16640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.867+0000 7fcb85f16640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb801035e0 tx=0x7fcb78009460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:34.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:34.871+0000 7fcb76ffd640 1 -- 172.21.15.19:0/3053591067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb700c84f0 con 0x7fcb80103c20 2024-04-25T10:52:35.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:34 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] up:active 2024-04-25T10:52:35.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:34 smithi169 ceph-mon[33049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:35.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:34 smithi169 ceph-mon[33049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:35.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:34 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1590975050' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T10:52:35.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:34 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] up:active 2024-04-25T10:52:35.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:34 smithi112 ceph-mon[32799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:35.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:34 smithi112 ceph-mon[32799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:35.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:34 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1590975050' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T10:52:35.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.096+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fcb801a2aa0 con 0x7fcb80103c20 2024-04-25T10:52:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:34 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] up:active 2024-04-25T10:52:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:34 smithi019 ceph-mon[29209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:34 smithi019 ceph-mon[29209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:34 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1590975050' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T10:52:35.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.735+0000 7fcb76ffd640 1 -- 172.21.15.19:0/3053591067 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fcb70003e60 con 0x7fcb80103c20 2024-04-25T10:52:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb6c0afaf0 msgr2=0x7fcb6c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb801035e0 tx=0x7fcb78009460 comp rx=0 tx=0).stop 2024-04-25T10:52:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 msgr2=0x7fcb8019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb8019aac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006010 tx=0x7fcb70004340 comp rx=0 tx=0).stop 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 shutdown_connections 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80108920 0x7fcb801a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb80103c20 0x7fcb8019aac0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 --2- 172.21.15.19:0/3053591067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb801029e0 0x7fcb8019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 >> 172.21.15.19:0/3053591067 conn(0x7fcb800fdbd0 msgr2=0x7fcb80109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 shutdown_connections 2024-04-25T10:52:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.738+0000 7fcb8817a640 1 -- 172.21.15.19:0/3053591067 wait complete. 2024-04-25T10:52:35.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T10:52:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 -- 172.21.15.19:0/2498102139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 msgr2=0x7fd7881036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 --2- 172.21.15.19:0/2498102139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd7881036e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009900 tx=0x7fd77c02a410 comp rx=0 tx=0).stop 2024-04-25T10:52:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 -- 172.21.15.19:0/2498102139 shutdown_connections 2024-04-25T10:52:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 --2- 172.21.15.19:0/2498102139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd788108920 0x7fd78810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 --2- 172.21.15.19:0/2498102139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd788103c20 0x7fd788104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 --2- 172.21.15.19:0/2498102139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd7881036e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 -- 172.21.15.19:0/2498102139 >> 172.21.15.19:0/2498102139 conn(0x7fd7880fdbd0 msgr2=0x7fd788100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 -- 172.21.15.19:0/2498102139 shutdown_connections 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 -- 172.21.15.19:0/2498102139 wait complete. 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.876+0000 7fd78f734640 1 Processor -- start 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 -- start start 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd78819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd788103c20 0x7fd78819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd788108920 0x7fd7881a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810f610 con 0x7fd7881029e0 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810f490 con 0x7fd788103c20 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810f790 con 0x7fd788108920 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd78819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd78819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41200/0 (socket says 172.21.15.19:41200) 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 -- 172.21.15.19:0/3801387604 learned_addr learned my addr 172.21.15.19:0/3801387604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78cccf640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd788103c20 0x7fd78819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 -- 172.21.15.19:0/3801387604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd788108920 msgr2=0x7fd7881a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd788108920 0x7fd7881a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 -- 172.21.15.19:0/3801387604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd788103c20 msgr2=0x7fd78819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd788103c20 0x7fd78819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 -- 172.21.15.19:0/3801387604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7881a1fe0 con 0x7fd7881029e0 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78cccf640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd788103c20 0x7fd78819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78d4d0640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd78819a550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd77c006010 tx=0x7fd77c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd77a7fc640 1 -- 172.21.15.19:0/3801387604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c03f030 con 0x7fd7881029e0 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd77a7fc640 1 -- 172.21.15.19:0/3801387604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd77c003aa0 con 0x7fd7881029e0 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.877+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7881a2270 con 0x7fd7881029e0 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.878+0000 7fd77a7fc640 1 -- 172.21.15.19:0/3801387604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c035070 con 0x7fd7881029e0 2024-04-25T10:52:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.878+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7881a27b0 con 0x7fd7881029e0 2024-04-25T10:52:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.878+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd788107730 con 0x7fd7881029e0 2024-04-25T10:52:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.880+0000 7fd77a7fc640 1 -- 172.21.15.19:0/3801387604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd77c042030 con 0x7fd7881029e0 2024-04-25T10:52:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.881+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd7740afb70 0x7fd7740b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.881+0000 7fd78cccf640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd7740afb70 0x7fd7740b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.881+0000 7fd77a7fc640 1 -- 172.21.15.19:0/3801387604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd77c0ff580 con 0x7fd7881029e0 2024-04-25T10:52:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.881+0000 7fd78cccf640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd7740afb70 0x7fd7740b2080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd78819b5f0 tx=0x7fd784005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:35.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:35.884+0000 7fd77a7fc640 1 -- 172.21.15.19:0/3801387604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd77c0c8210 con 0x7fd7881029e0 2024-04-25T10:52:36.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:35 smithi169 ceph-mon[33049]: pgmap v279: 97 pgs: 5 creating+activating, 25 creating+peering, 33 active+clean, 34 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-25T10:52:36.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:35 smithi169 ceph-mon[33049]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T10:52:36.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:35 smithi169 ceph-mon[33049]: Cluster is now healthy 2024-04-25T10:52:36.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:35 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1590975050' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T10:52:36.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:35 smithi169 ceph-mon[33049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:36.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:35 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3053591067' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T10:52:36.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:35 smithi112 ceph-mon[32799]: pgmap v279: 97 pgs: 5 creating+activating, 25 creating+peering, 33 active+clean, 34 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-25T10:52:36.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:35 smithi112 ceph-mon[32799]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T10:52:36.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:35 smithi112 ceph-mon[32799]: Cluster is now healthy 2024-04-25T10:52:36.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:35 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1590975050' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T10:52:36.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:35 smithi112 ceph-mon[32799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:36.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:35 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3053591067' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T10:52:36.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:35 smithi019 ceph-mon[29209]: pgmap v279: 97 pgs: 5 creating+activating, 25 creating+peering, 33 active+clean, 34 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-25T10:52:36.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:35 smithi019 ceph-mon[29209]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T10:52:36.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:35 smithi019 ceph-mon[29209]: Cluster is now healthy 2024-04-25T10:52:36.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:35 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1590975050' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T10:52:36.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:35 smithi019 ceph-mon[29209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:36.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:35 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3053591067' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T10:52:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.112+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd78802de90 con 0x7fd7881029e0 2024-04-25T10:52:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.745+0000 7fd77a7fc640 1 -- 172.21.15.19:0/3801387604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fd77c0c7b10 con 0x7fd7881029e0 2024-04-25T10:52:36.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd7740afb70 msgr2=0x7fd7740b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:36.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd7740afb70 0x7fd7740b2080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd78819b5f0 tx=0x7fd784005db0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 msgr2=0x7fd78819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd78819a550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd77c006010 tx=0x7fd77c031a50 comp rx=0 tx=0).stop 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 shutdown_connections 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd788108920 0x7fd7881a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd788103c20 0x7fd78819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd7740afb70 0x7fd7740b2080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 --2- 172.21.15.19:0/3801387604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7881029e0 0x7fd78819a550 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.748+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 >> 172.21.15.19:0/3801387604 conn(0x7fd7880fdbd0 msgr2=0x7fd788104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.749+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 shutdown_connections 2024-04-25T10:52:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.749+0000 7fd78f734640 1 -- 172.21.15.19:0/3801387604 wait complete. 2024-04-25T10:52:36.763 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-25T10:52:36.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 -- 172.21.15.19:0/2191671128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1201074c0 msgr2=0x7fb1201078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 --2- 172.21.15.19:0/2191671128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1201074c0 0x7fb1201078d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ca0 tx=0x7fb11802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 -- 172.21.15.19:0/2191671128 shutdown_connections 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 --2- 172.21.15.19:0/2191671128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1201074c0 0x7fb1201078d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 --2- 172.21.15.19:0/2191671128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb120104b60 0x7fb120106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 --2- 172.21.15.19:0/2191671128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120102200 0x7fb120104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 -- 172.21.15.19:0/2191671128 >> 172.21.15.19:0/2191671128 conn(0x7fb1200fda10 msgr2=0x7fb1200ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 -- 172.21.15.19:0/2191671128 shutdown_connections 2024-04-25T10:52:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.885+0000 7fb126d9d640 1 -- 172.21.15.19:0/2191671128 wait complete. 2024-04-25T10:52:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 Processor -- start 2024-04-25T10:52:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 -- start start 2024-04-25T10:52:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb120102200 0x7fb1201a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb120104b60 0x7fb1201a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201074c0 0x7fb1201a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb120115a40 con 0x7fb120102200 2024-04-25T10:52:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1201158c0 con 0x7fb1201074c0 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb126d9d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb120115bc0 con 0x7fb120104b60 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb120102200 0x7fb1201a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb120102200 0x7fb1201a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41206/0 (socket says 172.21.15.19:41206) 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 -- 172.21.15.19:0/562509057 learned_addr learned my addr 172.21.15.19:0/562509057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb12559a640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb120104b60 0x7fb1201a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb12659c640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201074c0 0x7fb1201a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 -- 172.21.15.19:0/562509057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb120104b60 msgr2=0x7fb1201a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb120104b60 0x7fb1201a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 -- 172.21.15.19:0/562509057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201074c0 msgr2=0x7fb1201a7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201074c0 0x7fb1201a7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb125d9b640 1 -- 172.21.15.19:0/562509057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1201a8180 con 0x7fb120102200 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb12659c640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201074c0 0x7fb1201a7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:52:36.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.886+0000 7fb12559a640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb120104b60 0x7fb1201a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:52:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.887+0000 7fb125d9b640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb120102200 0x7fb1201a0640 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d770 tx=0x7fb11c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.887+0000 7fb116ffd640 1 -- 172.21.15.19:0/562509057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c017070 con 0x7fb120102200 2024-04-25T10:52:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.887+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1201a8470 con 0x7fb120102200 2024-04-25T10:52:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.887+0000 7fb116ffd640 1 -- 172.21.15.19:0/562509057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb11c009c30 con 0x7fb120102200 2024-04-25T10:52:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.887+0000 7fb116ffd640 1 -- 172.21.15.19:0/562509057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c00f620 con 0x7fb120102200 2024-04-25T10:52:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.887+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1201a8a10 con 0x7fb120102200 2024-04-25T10:52:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.888+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1200ff530 con 0x7fb120102200 2024-04-25T10:52:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.889+0000 7fb116ffd640 1 -- 172.21.15.19:0/562509057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb11c01a070 con 0x7fb120102200 2024-04-25T10:52:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.890+0000 7fb116ffd640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb10c0afae0 0x7fb10c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.891+0000 7fb12559a640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb10c0afae0 0x7fb10c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.891+0000 7fb116ffd640 1 -- 172.21.15.19:0/562509057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fb11c0de6c0 con 0x7fb120102200 2024-04-25T10:52:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.891+0000 7fb12559a640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb10c0afae0 0x7fb10c0b1ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1201a16e0 tx=0x7fb110005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:36.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:36.894+0000 7fb116ffd640 1 -- 172.21.15.19:0/562509057 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb11c009de0 con 0x7fb120102200 2024-04-25T10:52:37.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:36 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3053591067' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T10:52:37.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:36 smithi169 ceph-mon[33049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:37.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:36 smithi169 ceph-mon[33049]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T10:52:37.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:36 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3801387604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T10:52:37.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:36 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3053591067' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T10:52:37.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:36 smithi112 ceph-mon[32799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:37.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:36 smithi112 ceph-mon[32799]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T10:52:37.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:36 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3801387604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T10:52:37.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.107+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb120062fe0 con 0x7fb10c0afae0 2024-04-25T10:52:37.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:36 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3053591067' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T10:52:37.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:36 smithi019 ceph-mon[29209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:52:37.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:36 smithi019 ceph-mon[29209]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T10:52:37.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:36 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3801387604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T10:52:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.130+0000 7fb116ffd640 1 -- 172.21.15.19:0/562509057 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb120062fe0 con 0x7fb10c0afae0 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.131+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb10c0afae0 msgr2=0x7fb10c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.131+0000 7fb126d9d640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb10c0afae0 0x7fb10c0b1ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1201a16e0 tx=0x7fb110005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb120102200 msgr2=0x7fb1201a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb120102200 0x7fb1201a0640 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d770 tx=0x7fb11c00dc30 comp rx=0 tx=0).stop 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 shutdown_connections 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201074c0 0x7fb1201a7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb120104b60 0x7fb1201a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fb10c0afae0 0x7fb10c0b1ff0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 --2- 172.21.15.19:0/562509057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb120102200 0x7fb1201a0640 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 >> 172.21.15.19:0/562509057 conn(0x7fb1200fda10 msgr2=0x7fb1201030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 shutdown_connections 2024-04-25T10:52:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.132+0000 7fb126d9d640 1 -- 172.21.15.19:0/562509057 wait complete. 2024-04-25T10:52:37.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T10:52:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 -- 172.21.15.19:0/3563132414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 msgr2=0x7f927810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 --2- 172.21.15.19:0/3563132414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f927810bcb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9270009ca0 tx=0x7f927002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 -- 172.21.15.19:0/3563132414 shutdown_connections 2024-04-25T10:52:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 --2- 172.21.15.19:0/3563132414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f927810bcb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 --2- 172.21.15.19:0/3563132414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9278106f30 0x7f9278109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 --2- 172.21.15.19:0/3563132414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92781045d0 0x7f92781069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 -- 172.21.15.19:0/3563132414 >> 172.21.15.19:0/3563132414 conn(0x7f92780fdbd0 msgr2=0x7f9278100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 -- 172.21.15.19:0/3563132414 shutdown_connections 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.260+0000 7f927ce68640 1 -- 172.21.15.19:0/3563132414 wait complete. 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 Processor -- start 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 -- start start 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92781045d0 0x7f927819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9278106f30 0x7f927819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f92781a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f927810f610 con 0x7f9278109890 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f927810f490 con 0x7f92781045d0 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927ce68640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f927810f790 con 0x7f9278106f30 2024-04-25T10:52:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f92781a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f92781a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41248/0 (socket says 172.21.15.19:41248) 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 -- 172.21.15.19:0/414090424 learned_addr learned my addr 172.21.15.19:0/414090424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9275d9b640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9278106f30 0x7f927819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927659c640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92781045d0 0x7f927819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 -- 172.21.15.19:0/414090424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9278106f30 msgr2=0x7f927819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9278106f30 0x7f927819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 -- 172.21.15.19:0/414090424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92781045d0 msgr2=0x7f927819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92781045d0 0x7f927819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9276d9d640 1 -- 172.21.15.19:0/414090424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92781a2080 con 0x7f9278109890 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f9275d9b640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9278106f30 0x7f927819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.261+0000 7f927659c640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92781045d0 0x7f927819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.262+0000 7f9276d9d640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f92781a1a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9270033040 tx=0x7f9270031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.262+0000 7f92677fe640 1 -- 172.21.15.19:0/414090424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270003b70 con 0x7f9278109890 2024-04-25T10:52:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.262+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92781a2310 con 0x7f9278109890 2024-04-25T10:52:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.262+0000 7f92677fe640 1 -- 172.21.15.19:0/414090424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270003d10 con 0x7f9278109890 2024-04-25T10:52:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.262+0000 7f92677fe640 1 -- 172.21.15.19:0/414090424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927002fab0 con 0x7f9278109890 2024-04-25T10:52:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.262+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92781a2860 con 0x7f9278109890 2024-04-25T10:52:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.263+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927806c2a0 con 0x7f9278109890 2024-04-25T10:52:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.265+0000 7f92677fe640 1 -- 172.21.15.19:0/414090424 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f927002fc50 con 0x7f9278109890 2024-04-25T10:52:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.265+0000 7f92677fe640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92600afaf0 0x7f92600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.266+0000 7f927659c640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92600afaf0 0x7f92600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.266+0000 7f92677fe640 1 -- 172.21.15.19:0/414090424 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f9270100410 con 0x7f9278109890 2024-04-25T10:52:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.266+0000 7f927659c640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92600afaf0 0x7f92600b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f927806a8b0 tx=0x7f9268005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.270+0000 7f92677fe640 1 -- 172.21.15.19:0/414090424 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92700c9140 con 0x7f9278109890 2024-04-25T10:52:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.488+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f92781a2b80 con 0x7f92600afaf0 2024-04-25T10:52:37.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.515+0000 7f92677fe640 1 -- 172.21.15.19:0/414090424 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92781a2b80 con 0x7f92600afaf0 2024-04-25T10:52:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92600afaf0 msgr2=0x7f92600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92600afaf0 0x7f92600b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f927806a8b0 tx=0x7f9268005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 msgr2=0x7f92781a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f92781a1a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9270033040 tx=0x7f9270031680 comp rx=0 tx=0).stop 2024-04-25T10:52:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 shutdown_connections 2024-04-25T10:52:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92600afaf0 0x7f92600b2000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9278109890 0x7f92781a1a10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9278106f30 0x7f927819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 --2- 172.21.15.19:0/414090424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92781045d0 0x7f927819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 >> 172.21.15.19:0/414090424 conn(0x7f92780fdbd0 msgr2=0x7f9278105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 shutdown_connections 2024-04-25T10:52:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.517+0000 7f927ce68640 1 -- 172.21.15.19:0/414090424 wait complete. 2024-04-25T10:52:37.528 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:52:37.113+0000 7f69ca0a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:52:37.529 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:52:37.113+0000 7f69ca0a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:52:37.529 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:52:37.113+0000 7f69ca0a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:52:37.529 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:52:37.113+0000 7f69ca0a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:52:37.529 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:52:37.113+0000 7f69ca0a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:52:37.530 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T10:52:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 -- 172.21.15.19:0/2610421159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 msgr2=0x7fd2a8105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 --2- 172.21.15.19:0/2610421159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a8105140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009900 tx=0x7fd29c02a410 comp rx=0 tx=0).stop 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 -- 172.21.15.19:0/2610421159 shutdown_connections 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 --2- 172.21.15.19:0/2610421159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8109a30 0x7fd2a810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 --2- 172.21.15.19:0/2610421159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a8105140 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 --2- 172.21.15.19:0/2610421159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a8103af0 0x7fd2a81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 -- 172.21.15.19:0/2610421159 >> 172.21.15.19:0/2610421159 conn(0x7fd2a80fece0 msgr2=0x7fd2a8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 -- 172.21.15.19:0/2610421159 shutdown_connections 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 -- 172.21.15.19:0/2610421159 wait complete. 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 Processor -- start 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.646+0000 7fd2ae667640 1 -- start start 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2ae667640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a8103af0 0x7fd2a819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2ae667640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2ae667640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8109a30 0x7fd2a81a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2ae667640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a810f620 con 0x7fd2a8104d30 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2ae667640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a810f4a0 con 0x7fd2a8103af0 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2ae667640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a810f7a0 con 0x7fd2a8109a30 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41260/0 (socket says 172.21.15.19:41260) 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 -- 172.21.15.19:0/1763603568 learned_addr learned my addr 172.21.15.19:0/1763603568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a7fff640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a8103af0 0x7fd2a819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2acc04640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8109a30 0x7fd2a81a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 -- 172.21.15.19:0/1763603568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8109a30 msgr2=0x7fd2a81a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8109a30 0x7fd2a81a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 -- 172.21.15.19:0/1763603568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a8103af0 msgr2=0x7fd2a819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a8103af0 0x7fd2a819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 -- 172.21.15.19:0/1763603568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81a1c70 con 0x7fd2a8104d30 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.647+0000 7fd2a77fe640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a819a720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd29c006010 tx=0x7fd29c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.648+0000 7fd2a57fa640 1 -- 172.21.15.19:0/1763603568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c03f030 con 0x7fd2a8104d30 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.648+0000 7fd2a57fa640 1 -- 172.21.15.19:0/1763603568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd29c003cb0 con 0x7fd2a8104d30 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.648+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a81a1f00 con 0x7fd2a8104d30 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.648+0000 7fd2a57fa640 1 -- 172.21.15.19:0/1763603568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c035070 con 0x7fd2a8104d30 2024-04-25T10:52:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.648+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a2310 con 0x7fd2a8104d30 2024-04-25T10:52:37.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.649+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a8108840 con 0x7fd2a8104d30 2024-04-25T10:52:37.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.650+0000 7fd2a57fa640 1 -- 172.21.15.19:0/1763603568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd29c042030 con 0x7fd2a8104d30 2024-04-25T10:52:37.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.651+0000 7fd2a57fa640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd2900afaf0 0x7fd2900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.651+0000 7fd2a7fff640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd2900afaf0 0x7fd2900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.651+0000 7fd2a57fa640 1 -- 172.21.15.19:0/1763603568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd29c0ff800 con 0x7fd2a8104d30 2024-04-25T10:52:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.652+0000 7fd2a7fff640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd2900afaf0 0x7fd2900b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2a81046f0 tx=0x7fd298005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:37.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.655+0000 7fd2a57fa640 1 -- 172.21.15.19:0/1763603568 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd29c0c8510 con 0x7fd2a8104d30 2024-04-25T10:52:37.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-mon[29209]: pgmap v280: 97 pgs: 5 creating+activating, 25 creating+peering, 52 active+clean, 15 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s wr, 5 op/s 2024-04-25T10:52:37.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3801387604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T10:52:37.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:37 smithi019 ceph-mon[29209]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T10:52:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.875+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd2a8062fe0 con 0x7fd2900afaf0 2024-04-25T10:52:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.881+0000 7fd2a57fa640 1 -- 172.21.15.19:0/1763603568 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd2a8062fe0 con 0x7fd2900afaf0 2024-04-25T10:52:37.882 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/bf62ce04-114d-44a1-bc79-7a0650c4964e 2024-04-25T10:52:37.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd2900afaf0 msgr2=0x7fd2900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd2900afaf0 0x7fd2900b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2a81046f0 tx=0x7fd298005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:37.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 msgr2=0x7fd2a819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:37.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a819a720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd29c006010 tx=0x7fd29c004340 comp rx=0 tx=0).stop 2024-04-25T10:52:37.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 shutdown_connections 2024-04-25T10:52:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8109a30 0x7fd2a81a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd2900afaf0 0x7fd2900b2000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a8104d30 0x7fd2a819a720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 --2- 172.21.15.19:0/1763603568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a8103af0 0x7fd2a819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 >> 172.21.15.19:0/1763603568 conn(0x7fd2a80fece0 msgr2=0x7fd2a8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 shutdown_connections 2024-04-25T10:52:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:37.884+0000 7fd2ae667640 1 -- 172.21.15.19:0/1763603568 wait complete. 2024-04-25T10:52:37.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T10:52:38.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:37 smithi169 ceph-mon[33049]: pgmap v280: 97 pgs: 5 creating+activating, 25 creating+peering, 52 active+clean, 15 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s wr, 5 op/s 2024-04-25T10:52:38.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:37 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3801387604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T10:52:38.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:37 smithi169 ceph-mon[33049]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 -- 172.21.15.19:0/2700617046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc106f30 msgr2=0x7f2cfc109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 --2- 172.21.15.19:0/2700617046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc106f30 0x7f2cfc109350 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009900 tx=0x7f2cf002a410 comp rx=0 tx=0).stop 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 -- 172.21.15.19:0/2700617046 shutdown_connections 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 --2- 172.21.15.19:0/2700617046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cfc109890 0x7f2cfc10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 --2- 172.21.15.19:0/2700617046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc106f30 0x7f2cfc109350 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 --2- 172.21.15.19:0/2700617046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc1045d0 0x7f2cfc1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 -- 172.21.15.19:0/2700617046 >> 172.21.15.19:0/2700617046 conn(0x7f2cfc0fdbd0 msgr2=0x7f2cfc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 -- 172.21.15.19:0/2700617046 shutdown_connections 2024-04-25T10:52:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.013+0000 7f2d00b86640 1 -- 172.21.15.19:0/2700617046 wait complete. 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 Processor -- start 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 -- start start 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1045d0 0x7f2cfc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc106f30 0x7f2cfc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cfc109890 0x7f2cfc1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10f680 con 0x7f2cfc1045d0 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10f500 con 0x7f2cfc106f30 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2d00b86640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10f800 con 0x7f2cfc109890 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1045d0 0x7f2cfc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1045d0 0x7f2cfc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41268/0 (socket says 172.21.15.19:41268) 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 -- 172.21.15.19:0/1887620125 learned_addr learned my addr 172.21.15.19:0/1887620125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:38.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cf9d9b640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc106f30 0x7f2cfc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 -- 172.21.15.19:0/1887620125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cfc109890 msgr2=0x7f2cfc1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cfc109890 0x7f2cfc1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 -- 172.21.15.19:0/1887620125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc106f30 msgr2=0x7f2cfc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc106f30 0x7f2cfc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cfa59c640 1 -- 172.21.15.19:0/1887620125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1a2100 con 0x7f2cfc1045d0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.014+0000 7f2cf9d9b640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc106f30 0x7f2cfc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.015+0000 7f2cfa59c640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1045d0 0x7f2cfc19a5c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c970 tx=0x7f2cec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.015+0000 7f2ceb7fe640 1 -- 172.21.15.19:0/1887620125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007bb0 con 0x7f2cfc1045d0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.015+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1a23f0 con 0x7f2cfc1045d0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.015+0000 7f2ceb7fe640 1 -- 172.21.15.19:0/1887620125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cec007d50 con 0x7f2cfc1045d0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.015+0000 7f2ceb7fe640 1 -- 172.21.15.19:0/1887620125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00f450 con 0x7f2cfc1045d0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.015+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1a29c0 con 0x7f2cfc1045d0 2024-04-25T10:52:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.016+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc06c2a0 con 0x7f2cfc1045d0 2024-04-25T10:52:38.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.018+0000 7f2ceb7fe640 1 -- 172.21.15.19:0/1887620125 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2cec004f20 con 0x7f2cfc1045d0 2024-04-25T10:52:38.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.018+0000 7f2ceb7fe640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ce40afaf0 0x7f2ce40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.018+0000 7f2cf9d9b640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ce40afaf0 0x7f2ce40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.018+0000 7f2ceb7fe640 1 -- 172.21.15.19:0/1887620125 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f2cec0debc0 con 0x7f2cfc1045d0 2024-04-25T10:52:38.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.019+0000 7f2cf9d9b640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ce40afaf0 0x7f2ce40b2000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0006010 tx=0x7f2cf0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:38.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.022+0000 7f2ceb7fe640 1 -- 172.21.15.19:0/1887620125 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cec014030 con 0x7f2cfc1045d0 2024-04-25T10:52:38.059 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 10:52:37 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b[62588]: 2024-04-25T10:52:37.663+0000 7fe3f2886700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:52:38.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:37 smithi112 ceph-mon[32799]: pgmap v280: 97 pgs: 5 creating+activating, 25 creating+peering, 52 active+clean, 15 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s wr, 5 op/s 2024-04-25T10:52:38.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:37 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3801387604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T10:52:38.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:37 smithi112 ceph-mon[32799]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T10:52:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.236+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2cfc1065d0 con 0x7f2ce40afaf0 2024-04-25T10:52:38.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.267+0000 7f2ceb7fe640 1 -- 172.21.15.19:0/1887620125 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2cfc1065d0 con 0x7f2ce40afaf0 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ce40afaf0 msgr2=0x7f2ce40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ce40afaf0 0x7f2ce40b2000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0006010 tx=0x7f2cf0000bc0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1045d0 msgr2=0x7f2cfc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1045d0 0x7f2cfc19a5c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c970 tx=0x7f2cec00ce30 comp rx=0 tx=0).stop 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 shutdown_connections 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cfc109890 0x7f2cfc1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc106f30 0x7f2cfc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ce40afaf0 0x7f2ce40b2000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 --2- 172.21.15.19:0/1887620125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1045d0 0x7f2cfc19a5c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 >> 172.21.15.19:0/1887620125 conn(0x7f2cfc0fdbd0 msgr2=0x7f2cfc105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:38.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 shutdown_connections 2024-04-25T10:52:38.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.269+0000 7f2d00b86640 1 -- 172.21.15.19:0/1887620125 wait complete. 2024-04-25T10:52:38.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T10:52:38.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 -- 172.21.15.19:0/2346650224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 msgr2=0x7feb8410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 --2- 172.21.15.19:0/2346650224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb8410bcb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009ca0 tx=0x7feb7c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 -- 172.21.15.19:0/2346650224 shutdown_connections 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 --2- 172.21.15.19:0/2346650224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb8410bcb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 --2- 172.21.15.19:0/2346650224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84106f30 0x7feb84109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 --2- 172.21.15.19:0/2346650224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb841045d0 0x7feb841069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 -- 172.21.15.19:0/2346650224 >> 172.21.15.19:0/2346650224 conn(0x7feb840fdbd0 msgr2=0x7feb84100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 -- 172.21.15.19:0/2346650224 shutdown_connections 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.398+0000 7feb8bf72640 1 -- 172.21.15.19:0/2346650224 wait complete. 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 Processor -- start 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 -- start start 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb841045d0 0x7feb8419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84106f30 0x7feb8419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb841a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410f610 con 0x7feb84109890 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410f490 con 0x7feb841045d0 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8bf72640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410f790 con 0x7feb84106f30 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb841a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb841a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41284/0 (socket says 172.21.15.19:41284) 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 -- 172.21.15.19:0/682364923 learned_addr learned my addr 172.21.15.19:0/682364923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb89d0e640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb841045d0 0x7feb8419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8950d640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84106f30 0x7feb8419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 -- 172.21.15.19:0/682364923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84106f30 msgr2=0x7feb8419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84106f30 0x7feb8419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 -- 172.21.15.19:0/682364923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb841045d0 msgr2=0x7feb8419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb841045d0 0x7feb8419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb8a50f640 1 -- 172.21.15.19:0/682364923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb841a2080 con 0x7feb84109890 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.399+0000 7feb89d0e640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb841045d0 0x7feb8419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.400+0000 7feb8a50f640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb841a1a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009430 tx=0x7feb7c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.400+0000 7feb7affd640 1 -- 172.21.15.19:0/682364923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c02f380 con 0x7feb84109890 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.400+0000 7feb7affd640 1 -- 172.21.15.19:0/682364923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7c0317c0 con 0x7feb84109890 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.400+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb841a2310 con 0x7feb84109890 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.400+0000 7feb7affd640 1 -- 172.21.15.19:0/682364923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c02fcd0 con 0x7feb84109890 2024-04-25T10:52:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.400+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841a2860 con 0x7feb84109890 2024-04-25T10:52:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.401+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8406c2a0 con 0x7feb84109890 2024-04-25T10:52:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.402+0000 7feb7affd640 1 -- 172.21.15.19:0/682364923 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7feb7c004040 con 0x7feb84109890 2024-04-25T10:52:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.403+0000 7feb7affd640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feb700afc90 0x7feb700b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.403+0000 7feb89d0e640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feb700afc90 0x7feb700b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.403+0000 7feb7affd640 1 -- 172.21.15.19:0/682364923 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7feb7c1004a0 con 0x7feb84109890 2024-04-25T10:52:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.403+0000 7feb89d0e640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feb700afc90 0x7feb700b21a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb800058c0 tx=0x7feb80005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:38.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.406+0000 7feb7affd640 1 -- 172.21.15.19:0/682364923 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb7c0c9130 con 0x7feb84109890 2024-04-25T10:52:38.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.624+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7feb841a2ac0 con 0x7feb700afc90 2024-04-25T10:52:38.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.629+0000 7feb7affd640 1 -- 172.21.15.19:0/682364923 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7feb841a2ac0 con 0x7feb700afc90 2024-04-25T10:52:38.630 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/ffc04145-58c1-4c1c-87dd-774d8647033f 2024-04-25T10:52:38.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feb700afc90 msgr2=0x7feb700b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feb700afc90 0x7feb700b21a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb800058c0 tx=0x7feb80005850 comp rx=0 tx=0).stop 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 msgr2=0x7feb841a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb841a1a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009430 tx=0x7feb7c003ba0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 shutdown_connections 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feb700afc90 0x7feb700b21a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb84109890 0x7feb841a1a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84106f30 0x7feb8419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 --2- 172.21.15.19:0/682364923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb841045d0 0x7feb8419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.631+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 >> 172.21.15.19:0/682364923 conn(0x7feb840fdbd0 msgr2=0x7feb84105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.632+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 shutdown_connections 2024-04-25T10:52:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.632+0000 7feb8bf72640 1 -- 172.21.15.19:0/682364923 wait complete. 2024-04-25T10:52:38.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:52:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 -- 172.21.15.19:0/1126976026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024109750 msgr2=0x7f802410bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 --2- 172.21.15.19:0/1126976026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024109750 0x7f802410bc60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f801c009ca0 tx=0x7f801c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 -- 172.21.15.19:0/1126976026 shutdown_connections 2024-04-25T10:52:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 --2- 172.21.15.19:0/1126976026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024109750 0x7f802410bc60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 --2- 172.21.15.19:0/1126976026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8024106be0 0x7f80241090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 --2- 172.21.15.19:0/1126976026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241041f0 0x7f8024106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 -- 172.21.15.19:0/1126976026 >> 172.21.15.19:0/1126976026 conn(0x7f80240fdbd0 msgr2=0x7f8024100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 -- 172.21.15.19:0/1126976026 shutdown_connections 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.768+0000 7f802ba96640 1 -- 172.21.15.19:0/1126976026 wait complete. 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 Processor -- start 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 -- start start 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241041f0 0x7f802419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024106be0 0x7f802419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8024109750 0x7f80241a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f802410f5a0 con 0x7f8024106be0 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f802410f420 con 0x7f80241041f0 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802ba96640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f802410f720 con 0x7f8024109750 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f8029031640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024106be0 0x7f802419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f8029031640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024106be0 0x7f802419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41300/0 (socket says 172.21.15.19:41300) 2024-04-25T10:52:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f8029031640 1 -- 172.21.15.19:0/1403023294 learned_addr learned my addr 172.21.15.19:0/1403023294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f8029832640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241041f0 0x7f802419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.769+0000 7f802a033640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8024109750 0x7f80241a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f8029031640 1 -- 172.21.15.19:0/1403023294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8024109750 msgr2=0x7f80241a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f8029031640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8024109750 0x7f80241a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f8029031640 1 -- 172.21.15.19:0/1403023294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241041f0 msgr2=0x7f802419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f8029031640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241041f0 0x7f802419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f8029031640 1 -- 172.21.15.19:0/1403023294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80241a1c90 con 0x7f8024106be0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f8029031640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024106be0 0x7f802419a690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9f0 tx=0x7f801400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f801affd640 1 -- 172.21.15.19:0/1403023294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400cd80 con 0x7f8024106be0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f801affd640 1 -- 172.21.15.19:0/1403023294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80140108e0 con 0x7f8024106be0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80241a1f80 con 0x7f8024106be0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f801affd640 1 -- 172.21.15.19:0/1403023294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014018980 con 0x7f8024106be0 2024-04-25T10:52:38.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.770+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241a23f0 con 0x7f8024106be0 2024-04-25T10:52:38.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.771+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80240ff6f0 con 0x7f8024106be0 2024-04-25T10:52:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.772+0000 7f801affd640 1 -- 172.21.15.19:0/1403023294 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8014010430 con 0x7f8024106be0 2024-04-25T10:52:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.773+0000 7f801affd640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f80100afb80 0x7f80100b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.773+0000 7f8029832640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f80100afb80 0x7f80100b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.773+0000 7f801affd640 1 -- 172.21.15.19:0/1403023294 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f80140df460 con 0x7f8024106be0 2024-04-25T10:52:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.774+0000 7f8029832640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f80100afb80 0x7f80100b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f802406a850 tx=0x7f8020005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:38.777+0000 7f801affd640 1 -- 172.21.15.19:0/1403023294 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80140a79c0 con 0x7f8024106be0 2024-04-25T10:52:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.003+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802402db50 con 0x7f8024106be0 2024-04-25T10:52:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.003+0000 7f801affd640 1 -- 172.21.15.19:0/1403023294 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f80140b0090 con 0x7f8024106be0 2024-04-25T10:52:39.004 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:39.004 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:52:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:52:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f80100afb80 msgr2=0x7f80100b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f80100afb80 0x7f80100b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f802406a850 tx=0x7f8020005f90 comp rx=0 tx=0).stop 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024106be0 msgr2=0x7f802419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024106be0 0x7f802419a690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9f0 tx=0x7f801400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 shutdown_connections 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8024109750 0x7f80241a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f80100afb80 0x7f80100b2090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8024106be0 0x7f802419a690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 --2- 172.21.15.19:0/1403023294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241041f0 0x7f802419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 >> 172.21.15.19:0/1403023294 conn(0x7f80240fdbd0 msgr2=0x7f8024104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 shutdown_connections 2024-04-25T10:52:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.006+0000 7f802ba96640 1 -- 172.21.15.19:0/1403023294 wait complete. 2024-04-25T10:52:39.017 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:38 smithi019 ceph-mon[29209]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:39.017 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:38 smithi019 ceph-mon[29209]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:39.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:52:39.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:38 smithi112 ceph-mon[32799]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:39.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:38 smithi112 ceph-mon[32799]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 -- 172.21.15.19:0/2105221260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf0102180 msgr2=0x7f8bf0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 --2- 172.21.15.19:0/2105221260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf0102180 0x7f8bf0102e80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009900 tx=0x7f8bd802a410 comp rx=0 tx=0).stop 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 -- 172.21.15.19:0/2105221260 shutdown_connections 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 --2- 172.21.15.19:0/2105221260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bf01080c0 0x7f8bf010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 --2- 172.21.15.19:0/2105221260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf01033c0 0x7f8bf01037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 --2- 172.21.15.19:0/2105221260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf0102180 0x7f8bf0102e80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 -- 172.21.15.19:0/2105221260 >> 172.21.15.19:0/2105221260 conn(0x7f8bf00fd370 msgr2=0x7f8bf00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 -- 172.21.15.19:0/2105221260 shutdown_connections 2024-04-25T10:52:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 -- 172.21.15.19:0/2105221260 wait complete. 2024-04-25T10:52:39.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.129+0000 7f8bf5434640 1 Processor -- start 2024-04-25T10:52:39.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bf5434640 1 -- start start 2024-04-25T10:52:39.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bf5434640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bf0102180 0x7f8bf019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bf5434640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf01033c0 0x7f8bf019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bf5434640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf01080c0 0x7f8bf01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bf5434640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf010d3d0 con 0x7f8bf01080c0 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bf5434640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf010d250 con 0x7f8bf0102180 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bf5434640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf010d550 con 0x7f8bf01033c0 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf01080c0 0x7f8bf01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf01080c0 0x7f8bf01a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41312/0 (socket says 172.21.15.19:41312) 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 -- 172.21.15.19:0/257813967 learned_addr learned my addr 172.21.15.19:0/257813967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8beeffd640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bf0102180 0x7f8bf019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bee7fc640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf01033c0 0x7f8bf019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 -- 172.21.15.19:0/257813967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf01033c0 msgr2=0x7f8bf019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf01033c0 0x7f8bf019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 -- 172.21.15.19:0/257813967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bf0102180 msgr2=0x7f8bf019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bf0102180 0x7f8bf019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.130+0000 7f8bef7fe640 1 -- 172.21.15.19:0/257813967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf01a1d50 con 0x7f8bf01080c0 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.131+0000 7f8bef7fe640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf01080c0 0x7f8bf01a16e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8be800ec60 tx=0x7f8be800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.131+0000 7f8be7fff640 1 -- 172.21.15.19:0/257813967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8004d20 con 0x7f8bf01080c0 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.131+0000 7f8be7fff640 1 -- 172.21.15.19:0/257813967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be800ba20 con 0x7f8bf01080c0 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.131+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf01a1fe0 con 0x7f8bf01080c0 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.131+0000 7f8be7fff640 1 -- 172.21.15.19:0/257813967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be80056c0 con 0x7f8bf01080c0 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.131+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf01a24f0 con 0x7f8bf01080c0 2024-04-25T10:52:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.132+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf0106ed0 con 0x7f8bf01080c0 2024-04-25T10:52:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.134+0000 7f8be7fff640 1 -- 172.21.15.19:0/257813967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8be8005860 con 0x7f8bf01080c0 2024-04-25T10:52:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.135+0000 7f8be7fff640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bdc0afc00 0x7f8bdc0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.135+0000 7f8be7fff640 1 -- 172.21.15.19:0/257813967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f8be80df880 con 0x7f8bf01080c0 2024-04-25T10:52:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.135+0000 7f8beeffd640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bdc0afc00 0x7f8bdc0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.135+0000 7f8beeffd640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bdc0afc00 0x7f8bdc0b2110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8005ab0 tx=0x7f8bd8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.138+0000 7f8be7fff640 1 -- 172.21.15.19:0/257813967 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8be80a83b0 con 0x7f8bf01080c0 2024-04-25T10:52:39.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:38 smithi169 ceph-mon[33049]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:39.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:38 smithi169 ceph-mon[33049]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.329+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bf002dd20 con 0x7f8bf01080c0 2024-04-25T10:52:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.331+0000 7f8be7fff640 1 -- 172.21.15.19:0/257813967 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22226 (secure 0 0 0) 0x7f8be80a7cb0 con 0x7f8bf01080c0 2024-04-25T10:52:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:39.332 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":77,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:52:33.451613+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:52:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bdc0afc00 msgr2=0x7f8bdc0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bdc0afc00 0x7f8bdc0b2110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8005ab0 tx=0x7f8bd8031400 comp rx=0 tx=0).stop 2024-04-25T10:52:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf01080c0 msgr2=0x7f8bf01a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf01080c0 0x7f8bf01a16e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8be800ec60 tx=0x7f8be800c7a0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 shutdown_connections 2024-04-25T10:52:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bdc0afc00 0x7f8bdc0b2110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bf01080c0 0x7f8bf01a16e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf01033c0 0x7f8bf019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 --2- 172.21.15.19:0/257813967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bf0102180 0x7f8bf019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 >> 172.21.15.19:0/257813967 conn(0x7f8bf00fd370 msgr2=0x7f8bf01040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 shutdown_connections 2024-04-25T10:52:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.333+0000 7f8bf5434640 1 -- 172.21.15.19:0/257813967 wait complete. 2024-04-25T10:52:39.346 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:52:39.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:52:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 -- 172.21.15.19:0/1776941504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 msgr2=0x7fe230103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 --2- 172.21.15.19:0/1776941504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe230103700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe220009900 tx=0x7fe22002a410 comp rx=0 tx=0).stop 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 -- 172.21.15.19:0/1776941504 shutdown_connections 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 --2- 172.21.15.19:0/1776941504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe230108940 0x7fe23010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 --2- 172.21.15.19:0/1776941504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe230103c40 0x7fe230104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 --2- 172.21.15.19:0/1776941504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe230103700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 -- 172.21.15.19:0/1776941504 >> 172.21.15.19:0/1776941504 conn(0x7fe2300fdbf0 msgr2=0x7fe2301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 -- 172.21.15.19:0/1776941504 shutdown_connections 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.457+0000 7fe234c92640 1 -- 172.21.15.19:0/1776941504 wait complete. 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 Processor -- start 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 -- start start 2024-04-25T10:52:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe23019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe230103c40 0x7fe23019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe230108940 0x7fe2301a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe23010e4e0 con 0x7fe230102a00 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe23010e360 con 0x7fe230108940 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe234c92640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe23010e660 con 0x7fe230103c40 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe23019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe23019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41332/0 (socket says 172.21.15.19:41332) 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 -- 172.21.15.19:0/858423052 learned_addr learned my addr 172.21.15.19:0/858423052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 -- 172.21.15.19:0/858423052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe230103c40 msgr2=0x7fe23019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22ed9d640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe230108940 0x7fe2301a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22dd9b640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe230103c40 0x7fe23019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe230103c40 0x7fe23019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 -- 172.21.15.19:0/858423052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe230108940 msgr2=0x7fe2301a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe230108940 0x7fe2301a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22e59c640 1 -- 172.21.15.19:0/858423052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2301a1cd0 con 0x7fe230102a00 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.458+0000 7fe22ed9d640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe230108940 0x7fe2301a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.459+0000 7fe22e59c640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe23019a240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe220032fd0 tx=0x7fe2200042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.459+0000 7fe21f7fe640 1 -- 172.21.15.19:0/858423052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22003f070 con 0x7fe230102a00 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.459+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2301a1f60 con 0x7fe230102a00 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.459+0000 7fe21f7fe640 1 -- 172.21.15.19:0/858423052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2200044d0 con 0x7fe230102a00 2024-04-25T10:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.459+0000 7fe21f7fe640 1 -- 172.21.15.19:0/858423052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220035070 con 0x7fe230102a00 2024-04-25T10:52:39.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.459+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2301a2360 con 0x7fe230102a00 2024-04-25T10:52:39.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.460+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2301a2650 con 0x7fe230102a00 2024-04-25T10:52:39.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.461+0000 7fe21f7fe640 1 -- 172.21.15.19:0/858423052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe22003b5f0 con 0x7fe230102a00 2024-04-25T10:52:39.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.462+0000 7fe21f7fe640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe2180afb80 0x7fe2180b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.462+0000 7fe22dd9b640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe2180afb80 0x7fe2180b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.462+0000 7fe21f7fe640 1 -- 172.21.15.19:0/858423052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fe2200ff280 con 0x7fe230102a00 2024-04-25T10:52:39.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.462+0000 7fe22dd9b640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe2180afb80 0x7fe2180b2090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2240058c0 tx=0x7fe224005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.467+0000 7fe21f7fe640 1 -- 172.21.15.19:0/858423052 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2200c7eb0 con 0x7fe230102a00 2024-04-25T10:52:39.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.651+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe23010a5f0 con 0x7fe2180afb80 2024-04-25T10:52:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.659+0000 7fe21f7fe640 1 -- 172.21.15.19:0/858423052 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258689 (secure 0 0 0) 0x7fe23010a5f0 con 0x7fe2180afb80 2024-04-25T10:52:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:52:39.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe2180afb80 msgr2=0x7fe2180b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe2180afb80 0x7fe2180b2090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2240058c0 tx=0x7fe224005c80 comp rx=0 tx=0).stop 2024-04-25T10:52:39.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 msgr2=0x7fe23019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe23019a240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe220032fd0 tx=0x7fe2200042e0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 shutdown_connections 2024-04-25T10:52:39.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe230108940 0x7fe2301a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe230103c40 0x7fe23019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe2180afb80 0x7fe2180b2090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 --2- 172.21.15.19:0/858423052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe230102a00 0x7fe23019a240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 >> 172.21.15.19:0/858423052 conn(0x7fe2300fdbf0 msgr2=0x7fe230109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:39.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 shutdown_connections 2024-04-25T10:52:39.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.662+0000 7fe234c92640 1 -- 172.21.15.19:0/858423052 wait complete. 2024-04-25T10:52:39.674 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-04-25T10:52:37.993699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":86,"num_read_kb":74,"num_write":168,"num_write_kb":2164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364840,"kb_used_data":11436,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576984,"statfs":{"total":1151940427776,"available":1151566831616,"internally_reserved":0,"allocated":11710464,"data_stored":6506898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.313145"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154931+0000","last_change":"2024-04-25T10:52:32.474738+0000","last_active":"2024-04-25T10:52:34.154931+0000","last_peered":"2024-04-25T10:52:34.154931+0000","last_clean":"2024-04-25T10:52:34.154931+0000","last_became_active":"2024-04-25T10:52:32.473812+0000","last_became_peered":"2024-04-25T10:52:32.473812+0000","last_unstale":"2024-04-25T10:52:34.154931+0000","last_undegraded":"2024-04-25T10:52:34.154931+0000","last_fullsized":"2024-04-25T10:52:34.154931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:25.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348904+0000","last_change":"2024-04-25T10:52:32.475491+0000","last_active":"2024-04-25T10:52:34.348904+0000","last_peered":"2024-04-25T10:52:34.348904+0000","last_clean":"2024-04-25T10:52:34.348904+0000","last_became_active":"2024-04-25T10:52:32.474164+0000","last_became_peered":"2024-04-25T10:52:32.474164+0000","last_unstale":"2024-04-25T10:52:34.348904+0000","last_undegraded":"2024-04-25T10:52:34.348904+0000","last_fullsized":"2024-04-25T10:52:34.348904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:58.838292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348730+0000","last_change":"2024-04-25T10:52:32.471349+0000","last_active":"2024-04-25T10:52:34.348730+0000","last_peered":"2024-04-25T10:52:34.348730+0000","last_clean":"2024-04-25T10:52:34.348730+0000","last_became_active":"2024-04-25T10:52:32.470972+0000","last_became_peered":"2024-04-25T10:52:32.470972+0000","last_unstale":"2024-04-25T10:52:34.348730+0000","last_undegraded":"2024-04-25T10:52:34.348730+0000","last_fullsized":"2024-04-25T10:52:34.348730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:28.634537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456431+0000","last_change":"2024-04-25T10:52:32.470212+0000","last_active":"2024-04-25T10:52:33.456431+0000","last_peered":"2024-04-25T10:52:33.456431+0000","last_clean":"2024-04-25T10:52:33.456431+0000","last_became_active":"2024-04-25T10:52:32.469642+0000","last_became_peered":"2024-04-25T10:52:32.469642+0000","last_unstale":"2024-04-25T10:52:33.456431+0000","last_undegraded":"2024-04-25T10:52:33.456431+0000","last_fullsized":"2024-04-25T10:52:33.456431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:08.231679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456367+0000","last_change":"2024-04-25T10:52:32.456568+0000","last_active":"2024-04-25T10:52:33.456367+0000","last_peered":"2024-04-25T10:52:33.456367+0000","last_clean":"2024-04-25T10:52:33.456367+0000","last_became_active":"2024-04-25T10:52:32.456051+0000","last_became_peered":"2024-04-25T10:52:32.456051+0000","last_unstale":"2024-04-25T10:52:33.456367+0000","last_undegraded":"2024-04-25T10:52:33.456367+0000","last_fullsized":"2024-04-25T10:52:33.456367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154170+0000","last_change":"2024-04-25T10:52:32.479781+0000","last_active":"2024-04-25T10:52:34.154170+0000","last_peered":"2024-04-25T10:52:34.154170+0000","last_clean":"2024-04-25T10:52:34.154170+0000","last_became_active":"2024-04-25T10:52:32.479213+0000","last_became_peered":"2024-04-25T10:52:32.479213+0000","last_unstale":"2024-04-25T10:52:34.154170+0000","last_undegraded":"2024-04-25T10:52:34.154170+0000","last_fullsized":"2024-04-25T10:52:34.154170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:16.239969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154843+0000","last_change":"2024-04-25T10:52:32.471430+0000","last_active":"2024-04-25T10:52:34.154843+0000","last_peered":"2024-04-25T10:52:34.154843+0000","last_clean":"2024-04-25T10:52:34.154843+0000","last_became_active":"2024-04-25T10:52:32.470518+0000","last_became_peered":"2024-04-25T10:52:32.470518+0000","last_unstale":"2024-04-25T10:52:34.154843+0000","last_undegraded":"2024-04-25T10:52:34.154843+0000","last_fullsized":"2024-04-25T10:52:34.154843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:12.846174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.714871+0000","last_change":"2024-04-25T10:52:32.472545+0000","last_active":"2024-04-25T10:52:33.714871+0000","last_peered":"2024-04-25T10:52:33.714871+0000","last_clean":"2024-04-25T10:52:33.714871+0000","last_became_active":"2024-04-25T10:52:32.472151+0000","last_became_peered":"2024-04-25T10:52:32.472151+0000","last_unstale":"2024-04-25T10:52:33.714871+0000","last_undegraded":"2024-04-25T10:52:33.714871+0000","last_fullsized":"2024-04-25T10:52:33.714871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:36.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154239+0000","last_change":"2024-04-25T10:52:32.466223+0000","last_active":"2024-04-25T10:52:34.154239+0000","last_peered":"2024-04-25T10:52:34.154239+0000","last_clean":"2024-04-25T10:52:34.154239+0000","last_became_active":"2024-04-25T10:52:32.465498+0000","last_became_peered":"2024-04-25T10:52:32.465498+0000","last_unstale":"2024-04-25T10:52:34.154239+0000","last_undegraded":"2024-04-25T10:52:34.154239+0000","last_fullsized":"2024-04-25T10:52:34.154239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:25.924450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154635+0000","last_change":"2024-04-25T10:52:32.470601+0000","last_active":"2024-04-25T10:52:34.154635+0000","last_peered":"2024-04-25T10:52:34.154635+0000","last_clean":"2024-04-25T10:52:34.154635+0000","last_became_active":"2024-04-25T10:52:32.468984+0000","last_became_peered":"2024-04-25T10:52:32.468984+0000","last_unstale":"2024-04-25T10:52:34.154635+0000","last_undegraded":"2024-04-25T10:52:34.154635+0000","last_fullsized":"2024-04-25T10:52:34.154635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155296+0000","last_change":"2024-04-25T10:52:32.463907+0000","last_active":"2024-04-25T10:52:34.155296+0000","last_peered":"2024-04-25T10:52:34.155296+0000","last_clean":"2024-04-25T10:52:34.155296+0000","last_became_active":"2024-04-25T10:52:32.463536+0000","last_became_peered":"2024-04-25T10:52:32.463536+0000","last_unstale":"2024-04-25T10:52:34.155296+0000","last_undegraded":"2024-04-25T10:52:34.155296+0000","last_fullsized":"2024-04-25T10:52:34.155296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.503805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456424+0000","last_change":"2024-04-25T10:52:32.473646+0000","last_active":"2024-04-25T10:52:33.456424+0000","last_peered":"2024-04-25T10:52:33.456424+0000","last_clean":"2024-04-25T10:52:33.456424+0000","last_became_active":"2024-04-25T10:52:32.473310+0000","last_became_peered":"2024-04-25T10:52:32.473310+0000","last_unstale":"2024-04-25T10:52:33.456424+0000","last_undegraded":"2024-04-25T10:52:33.456424+0000","last_fullsized":"2024-04-25T10:52:33.456424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.691293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791849+0000","last_change":"2024-04-25T10:52:32.473896+0000","last_active":"2024-04-25T10:52:33.791849+0000","last_peered":"2024-04-25T10:52:33.791849+0000","last_clean":"2024-04-25T10:52:33.791849+0000","last_became_active":"2024-04-25T10:52:32.473474+0000","last_became_peered":"2024-04-25T10:52:32.473474+0000","last_unstale":"2024-04-25T10:52:33.791849+0000","last_undegraded":"2024-04-25T10:52:33.791849+0000","last_fullsized":"2024-04-25T10:52:33.791849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:41.568301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348566+0000","last_change":"2024-04-25T10:52:32.476104+0000","last_active":"2024-04-25T10:52:34.348566+0000","last_peered":"2024-04-25T10:52:34.348566+0000","last_clean":"2024-04-25T10:52:34.348566+0000","last_became_active":"2024-04-25T10:52:32.475342+0000","last_became_peered":"2024-04-25T10:52:32.475342+0000","last_unstale":"2024-04-25T10:52:34.348566+0000","last_undegraded":"2024-04-25T10:52:34.348566+0000","last_fullsized":"2024-04-25T10:52:34.348566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:37.087793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348397+0000","last_change":"2024-04-25T10:52:32.470263+0000","last_active":"2024-04-25T10:52:34.348397+0000","last_peered":"2024-04-25T10:52:34.348397+0000","last_clean":"2024-04-25T10:52:34.348397+0000","last_became_active":"2024-04-25T10:52:32.469873+0000","last_became_peered":"2024-04-25T10:52:32.469873+0000","last_unstale":"2024-04-25T10:52:34.348397+0000","last_undegraded":"2024-04-25T10:52:34.348397+0000","last_fullsized":"2024-04-25T10:52:34.348397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.349598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153344+0000","last_change":"2024-04-25T10:52:32.471851+0000","last_active":"2024-04-25T10:52:34.153344+0000","last_peered":"2024-04-25T10:52:34.153344+0000","last_clean":"2024-04-25T10:52:34.153344+0000","last_became_active":"2024-04-25T10:52:32.471477+0000","last_became_peered":"2024-04-25T10:52:32.471477+0000","last_unstale":"2024-04-25T10:52:34.153344+0000","last_undegraded":"2024-04-25T10:52:34.153344+0000","last_fullsized":"2024-04-25T10:52:34.153344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:59.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154824+0000","last_change":"2024-04-25T10:52:32.472530+0000","last_active":"2024-04-25T10:52:34.154824+0000","last_peered":"2024-04-25T10:52:34.154824+0000","last_clean":"2024-04-25T10:52:34.154824+0000","last_became_active":"2024-04-25T10:52:32.472174+0000","last_became_peered":"2024-04-25T10:52:32.472174+0000","last_unstale":"2024-04-25T10:52:34.154824+0000","last_undegraded":"2024-04-25T10:52:34.154824+0000","last_fullsized":"2024-04-25T10:52:34.154824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:09.473575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455379+0000","last_change":"2024-04-25T10:52:32.473617+0000","last_active":"2024-04-25T10:52:33.455379+0000","last_peered":"2024-04-25T10:52:33.455379+0000","last_clean":"2024-04-25T10:52:33.455379+0000","last_became_active":"2024-04-25T10:52:32.473261+0000","last_became_peered":"2024-04-25T10:52:32.473261+0000","last_unstale":"2024-04-25T10:52:33.455379+0000","last_undegraded":"2024-04-25T10:52:33.455379+0000","last_fullsized":"2024-04-25T10:52:33.455379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:05.568672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155275+0000","last_change":"2024-04-25T10:52:32.466296+0000","last_active":"2024-04-25T10:52:34.155275+0000","last_peered":"2024-04-25T10:52:34.155275+0000","last_clean":"2024-04-25T10:52:34.155275+0000","last_became_active":"2024-04-25T10:52:32.463153+0000","last_became_peered":"2024-04-25T10:52:32.463153+0000","last_unstale":"2024-04-25T10:52:34.155275+0000","last_undegraded":"2024-04-25T10:52:34.155275+0000","last_fullsized":"2024-04-25T10:52:34.155275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455290+0000","last_change":"2024-04-25T10:52:31.701949+0000","last_active":"2024-04-25T10:52:33.455290+0000","last_peered":"2024-04-25T10:52:33.455290+0000","last_clean":"2024-04-25T10:52:33.455290+0000","last_became_active":"2024-04-25T10:52:31.701508+0000","last_became_peered":"2024-04-25T10:52:31.701508+0000","last_unstale":"2024-04-25T10:52:33.455290+0000","last_undegraded":"2024-04-25T10:52:33.455290+0000","last_fullsized":"2024-04-25T10:52:33.455290+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:41.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715922+0000","last_change":"2024-04-25T10:52:32.468871+0000","last_active":"2024-04-25T10:52:33.715922+0000","last_peered":"2024-04-25T10:52:33.715922+0000","last_clean":"2024-04-25T10:52:33.715922+0000","last_became_active":"2024-04-25T10:52:32.468460+0000","last_became_peered":"2024-04-25T10:52:32.468460+0000","last_unstale":"2024-04-25T10:52:33.715922+0000","last_undegraded":"2024-04-25T10:52:33.715922+0000","last_fullsized":"2024-04-25T10:52:33.715922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:29.055639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.716131+0000","last_change":"2024-04-25T10:52:31.705068+0000","last_active":"2024-04-25T10:52:33.716131+0000","last_peered":"2024-04-25T10:52:33.716131+0000","last_clean":"2024-04-25T10:52:33.716131+0000","last_became_active":"2024-04-25T10:52:31.704439+0000","last_became_peered":"2024-04-25T10:52:31.704439+0000","last_unstale":"2024-04-25T10:52:33.716131+0000","last_undegraded":"2024-04-25T10:52:33.716131+0000","last_fullsized":"2024-04-25T10:52:33.716131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.347653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348851+0000","last_change":"2024-04-25T10:52:32.476025+0000","last_active":"2024-04-25T10:52:34.348851+0000","last_peered":"2024-04-25T10:52:34.348851+0000","last_clean":"2024-04-25T10:52:34.348851+0000","last_became_active":"2024-04-25T10:52:32.475611+0000","last_became_peered":"2024-04-25T10:52:32.475611+0000","last_unstale":"2024-04-25T10:52:34.348851+0000","last_undegraded":"2024-04-25T10:52:34.348851+0000","last_fullsized":"2024-04-25T10:52:34.348851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:45.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456918+0000","last_change":"2024-04-25T10:52:31.703012+0000","last_active":"2024-04-25T10:52:33.456918+0000","last_peered":"2024-04-25T10:52:33.456918+0000","last_clean":"2024-04-25T10:52:33.456918+0000","last_became_active":"2024-04-25T10:52:31.702689+0000","last_became_peered":"2024-04-25T10:52:31.702689+0000","last_unstale":"2024-04-25T10:52:33.456918+0000","last_undegraded":"2024-04-25T10:52:33.456918+0000","last_fullsized":"2024-04-25T10:52:33.456918+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:13.038859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153493+0000","last_change":"2024-04-25T10:52:32.475858+0000","last_active":"2024-04-25T10:52:34.153493+0000","last_peered":"2024-04-25T10:52:34.153493+0000","last_clean":"2024-04-25T10:52:34.153493+0000","last_became_active":"2024-04-25T10:52:32.475486+0000","last_became_peered":"2024-04-25T10:52:32.475486+0000","last_unstale":"2024-04-25T10:52:34.153493+0000","last_undegraded":"2024-04-25T10:52:34.153493+0000","last_fullsized":"2024-04-25T10:52:34.153493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.439208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456913+0000","last_change":"2024-04-25T10:52:31.698229+0000","last_active":"2024-04-25T10:52:33.456913+0000","last_peered":"2024-04-25T10:52:33.456913+0000","last_clean":"2024-04-25T10:52:33.456913+0000","last_became_active":"2024-04-25T10:52:31.697771+0000","last_became_peered":"2024-04-25T10:52:31.697771+0000","last_unstale":"2024-04-25T10:52:33.456913+0000","last_undegraded":"2024-04-25T10:52:33.456913+0000","last_fullsized":"2024-04-25T10:52:33.456913+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:13.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154668+0000","last_change":"2024-04-25T10:52:32.466495+0000","last_active":"2024-04-25T10:52:34.154668+0000","last_peered":"2024-04-25T10:52:34.154668+0000","last_clean":"2024-04-25T10:52:34.154668+0000","last_became_active":"2024-04-25T10:52:32.462400+0000","last_became_peered":"2024-04-25T10:52:32.462400+0000","last_unstale":"2024-04-25T10:52:34.154668+0000","last_undegraded":"2024-04-25T10:52:34.154668+0000","last_fullsized":"2024-04-25T10:52:34.154668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:18.770981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154306+0000","last_change":"2024-04-25T10:52:31.699644+0000","last_active":"2024-04-25T10:52:34.154306+0000","last_peered":"2024-04-25T10:52:34.154306+0000","last_clean":"2024-04-25T10:52:34.154306+0000","last_became_active":"2024-04-25T10:52:31.699315+0000","last_became_peered":"2024-04-25T10:52:31.699315+0000","last_unstale":"2024-04-25T10:52:34.154306+0000","last_undegraded":"2024-04-25T10:52:34.154306+0000","last_fullsized":"2024-04-25T10:52:34.154306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:52.915096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456333+0000","last_change":"2024-04-25T10:52:32.463372+0000","last_active":"2024-04-25T10:52:33.456333+0000","last_peered":"2024-04-25T10:52:33.456333+0000","last_clean":"2024-04-25T10:52:33.456333+0000","last_became_active":"2024-04-25T10:52:32.462926+0000","last_became_peered":"2024-04-25T10:52:32.462926+0000","last_unstale":"2024-04-25T10:52:33.456333+0000","last_undegraded":"2024-04-25T10:52:33.456333+0000","last_fullsized":"2024-04-25T10:52:33.456333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:17.724268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153916+0000","last_change":"2024-04-25T10:52:31.695754+0000","last_active":"2024-04-25T10:52:34.153916+0000","last_peered":"2024-04-25T10:52:34.153916+0000","last_clean":"2024-04-25T10:52:34.153916+0000","last_became_active":"2024-04-25T10:52:31.695381+0000","last_became_peered":"2024-04-25T10:52:31.695381+0000","last_unstale":"2024-04-25T10:52:34.153916+0000","last_undegraded":"2024-04-25T10:52:34.153916+0000","last_fullsized":"2024-04-25T10:52:34.153916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:58.851534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154584+0000","last_change":"2024-04-25T10:52:32.461959+0000","last_active":"2024-04-25T10:52:34.154584+0000","last_peered":"2024-04-25T10:52:34.154584+0000","last_clean":"2024-04-25T10:52:34.154584+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:52:34.154584+0000","last_undegraded":"2024-04-25T10:52:34.154584+0000","last_fullsized":"2024-04-25T10:52:34.154584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:28.238824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791672+0000","last_change":"2024-04-25T10:52:31.695067+0000","last_active":"2024-04-25T10:52:33.791672+0000","last_peered":"2024-04-25T10:52:33.791672+0000","last_clean":"2024-04-25T10:52:33.791672+0000","last_became_active":"2024-04-25T10:52:31.694648+0000","last_became_peered":"2024-04-25T10:52:31.694648+0000","last_unstale":"2024-04-25T10:52:33.791672+0000","last_undegraded":"2024-04-25T10:52:33.791672+0000","last_fullsized":"2024-04-25T10:52:33.791672+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:09.654835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.714906+0000","last_change":"2024-04-25T10:52:32.453501+0000","last_active":"2024-04-25T10:52:33.714906+0000","last_peered":"2024-04-25T10:52:33.714906+0000","last_clean":"2024-04-25T10:52:33.714906+0000","last_became_active":"2024-04-25T10:52:32.453113+0000","last_became_peered":"2024-04-25T10:52:32.453113+0000","last_unstale":"2024-04-25T10:52:33.714906+0000","last_undegraded":"2024-04-25T10:52:33.714906+0000","last_fullsized":"2024-04-25T10:52:33.714906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:07.340709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153710+0000","last_change":"2024-04-25T10:52:31.697946+0000","last_active":"2024-04-25T10:52:34.153710+0000","last_peered":"2024-04-25T10:52:34.153710+0000","last_clean":"2024-04-25T10:52:34.153710+0000","last_became_active":"2024-04-25T10:52:31.697612+0000","last_became_peered":"2024-04-25T10:52:31.697612+0000","last_unstale":"2024-04-25T10:52:34.153710+0000","last_undegraded":"2024-04-25T10:52:34.153710+0000","last_fullsized":"2024-04-25T10:52:34.153710+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.280776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455450+0000","last_change":"2024-04-25T10:52:32.473537+0000","last_active":"2024-04-25T10:52:33.455450+0000","last_peered":"2024-04-25T10:52:33.455450+0000","last_clean":"2024-04-25T10:52:33.455450+0000","last_became_active":"2024-04-25T10:52:32.473164+0000","last_became_peered":"2024-04-25T10:52:32.473164+0000","last_unstale":"2024-04-25T10:52:33.455450+0000","last_undegraded":"2024-04-25T10:52:33.455450+0000","last_fullsized":"2024-04-25T10:52:33.455450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.008031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154421+0000","last_change":"2024-04-25T10:52:31.694207+0000","last_active":"2024-04-25T10:52:34.154421+0000","last_peered":"2024-04-25T10:52:34.154421+0000","last_clean":"2024-04-25T10:52:34.154421+0000","last_became_active":"2024-04-25T10:52:31.693777+0000","last_became_peered":"2024-04-25T10:52:31.693777+0000","last_unstale":"2024-04-25T10:52:34.154421+0000","last_undegraded":"2024-04-25T10:52:34.154421+0000","last_fullsized":"2024-04-25T10:52:34.154421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:43.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154422+0000","last_change":"2024-04-25T10:52:32.475545+0000","last_active":"2024-04-25T10:52:34.154422+0000","last_peered":"2024-04-25T10:52:34.154422+0000","last_clean":"2024-04-25T10:52:34.154422+0000","last_became_active":"2024-04-25T10:52:32.475195+0000","last_became_peered":"2024-04-25T10:52:32.475195+0000","last_unstale":"2024-04-25T10:52:34.154422+0000","last_undegraded":"2024-04-25T10:52:34.154422+0000","last_fullsized":"2024-04-25T10:52:34.154422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:49.900374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155036+0000","last_change":"2024-04-25T10:52:31.698184+0000","last_active":"2024-04-25T10:52:34.155036+0000","last_peered":"2024-04-25T10:52:34.155036+0000","last_clean":"2024-04-25T10:52:34.155036+0000","last_became_active":"2024-04-25T10:52:31.697816+0000","last_became_peered":"2024-04-25T10:52:31.697816+0000","last_unstale":"2024-04-25T10:52:34.155036+0000","last_undegraded":"2024-04-25T10:52:34.155036+0000","last_fullsized":"2024-04-25T10:52:34.155036+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:44.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348894+0000","last_change":"2024-04-25T10:52:32.472437+0000","last_active":"2024-04-25T10:52:34.348894+0000","last_peered":"2024-04-25T10:52:34.348894+0000","last_clean":"2024-04-25T10:52:34.348894+0000","last_became_active":"2024-04-25T10:52:32.471944+0000","last_became_peered":"2024-04-25T10:52:32.471944+0000","last_unstale":"2024-04-25T10:52:34.348894+0000","last_undegraded":"2024-04-25T10:52:34.348894+0000","last_fullsized":"2024-04-25T10:52:34.348894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:11.047158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154758+0000","last_change":"2024-04-25T10:52:31.696943+0000","last_active":"2024-04-25T10:52:34.154758+0000","last_peered":"2024-04-25T10:52:34.154758+0000","last_clean":"2024-04-25T10:52:34.154758+0000","last_became_active":"2024-04-25T10:52:31.696561+0000","last_became_peered":"2024-04-25T10:52:31.696561+0000","last_unstale":"2024-04-25T10:52:34.154758+0000","last_undegraded":"2024-04-25T10:52:34.154758+0000","last_fullsized":"2024-04-25T10:52:34.154758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:14.628800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153856+0000","last_change":"2024-04-25T10:52:32.461966+0000","last_active":"2024-04-25T10:52:34.153856+0000","last_peered":"2024-04-25T10:52:34.153856+0000","last_clean":"2024-04-25T10:52:34.153856+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:52:34.153856+0000","last_undegraded":"2024-04-25T10:52:34.153856+0000","last_fullsized":"2024-04-25T10:52:34.153856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'6","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154245+0000","last_change":"2024-04-25T10:52:31.690938+0000","last_active":"2024-04-25T10:52:34.154245+0000","last_peered":"2024-04-25T10:52:34.154245+0000","last_clean":"2024-04-25T10:52:34.154245+0000","last_became_active":"2024-04-25T10:52:31.690511+0000","last_became_peered":"2024-04-25T10:52:31.690511+0000","last_unstale":"2024-04-25T10:52:34.154245+0000","last_undegraded":"2024-04-25T10:52:34.154245+0000","last_fullsized":"2024-04-25T10:52:34.154245+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:10.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456758+0000","last_change":"2024-04-25T10:52:32.474871+0000","last_active":"2024-04-25T10:52:33.456758+0000","last_peered":"2024-04-25T10:52:33.456758+0000","last_clean":"2024-04-25T10:52:33.456758+0000","last_became_active":"2024-04-25T10:52:32.474463+0000","last_became_peered":"2024-04-25T10:52:32.474463+0000","last_unstale":"2024-04-25T10:52:33.456758+0000","last_undegraded":"2024-04-25T10:52:33.456758+0000","last_fullsized":"2024-04-25T10:52:33.456758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:36.719230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'2","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154653+0000","last_change":"2024-04-25T10:52:31.691006+0000","last_active":"2024-04-25T10:52:34.154653+0000","last_peered":"2024-04-25T10:52:34.154653+0000","last_clean":"2024-04-25T10:52:34.154653+0000","last_became_active":"2024-04-25T10:52:31.690611+0000","last_became_peered":"2024-04-25T10:52:31.690611+0000","last_unstale":"2024-04-25T10:52:34.154653+0000","last_undegraded":"2024-04-25T10:52:34.154653+0000","last_fullsized":"2024-04-25T10:52:34.154653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:37.781160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791737+0000","last_change":"2024-04-25T10:52:32.463331+0000","last_active":"2024-04-25T10:52:33.791737+0000","last_peered":"2024-04-25T10:52:33.791737+0000","last_clean":"2024-04-25T10:52:33.791737+0000","last_became_active":"2024-04-25T10:52:32.462579+0000","last_became_peered":"2024-04-25T10:52:32.462579+0000","last_unstale":"2024-04-25T10:52:33.791737+0000","last_undegraded":"2024-04-25T10:52:33.791737+0000","last_fullsized":"2024-04-25T10:52:33.791737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154357+0000","last_change":"2024-04-25T10:52:31.704675+0000","last_active":"2024-04-25T10:52:34.154357+0000","last_peered":"2024-04-25T10:52:34.154357+0000","last_clean":"2024-04-25T10:52:34.154357+0000","last_became_active":"2024-04-25T10:52:31.704236+0000","last_became_peered":"2024-04-25T10:52:31.704236+0000","last_unstale":"2024-04-25T10:52:34.154357+0000","last_undegraded":"2024-04-25T10:52:34.154357+0000","last_fullsized":"2024-04-25T10:52:34.154357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.637006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155385+0000","last_change":"2024-04-25T10:52:32.462960+0000","last_active":"2024-04-25T10:52:34.155385+0000","last_peered":"2024-04-25T10:52:34.155385+0000","last_clean":"2024-04-25T10:52:34.155385+0000","last_became_active":"2024-04-25T10:52:32.461993+0000","last_became_peered":"2024-04-25T10:52:32.461993+0000","last_unstale":"2024-04-25T10:52:34.155385+0000","last_undegraded":"2024-04-25T10:52:34.155385+0000","last_fullsized":"2024-04-25T10:52:34.155385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:01.803209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456118+0000","last_change":"2024-04-25T10:52:31.698206+0000","last_active":"2024-04-25T10:52:33.456118+0000","last_peered":"2024-04-25T10:52:33.456118+0000","last_clean":"2024-04-25T10:52:33.456118+0000","last_became_active":"2024-04-25T10:52:31.697814+0000","last_became_peered":"2024-04-25T10:52:31.697814+0000","last_unstale":"2024-04-25T10:52:33.456118+0000","last_undegraded":"2024-04-25T10:52:33.456118+0000","last_fullsized":"2024-04-25T10:52:33.456118+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:02.849229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154769+0000","last_change":"2024-04-25T10:52:32.473551+0000","last_active":"2024-04-25T10:52:34.154769+0000","last_peered":"2024-04-25T10:52:34.154769+0000","last_clean":"2024-04-25T10:52:34.154769+0000","last_became_active":"2024-04-25T10:52:32.473227+0000","last_became_peered":"2024-04-25T10:52:32.473227+0000","last_unstale":"2024-04-25T10:52:34.154769+0000","last_undegraded":"2024-04-25T10:52:34.154769+0000","last_fullsized":"2024-04-25T10:52:34.154769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:37.742066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'8","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456375+0000","last_change":"2024-04-25T10:52:31.700670+0000","last_active":"2024-04-25T10:52:33.456375+0000","last_peered":"2024-04-25T10:52:33.456375+0000","last_clean":"2024-04-25T10:52:33.456375+0000","last_became_active":"2024-04-25T10:52:31.700234+0000","last_became_peered":"2024-04-25T10:52:31.700234+0000","last_unstale":"2024-04-25T10:52:33.456375+0000","last_undegraded":"2024-04-25T10:52:33.456375+0000","last_fullsized":"2024-04-25T10:52:33.456375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:21.795784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154852+0000","last_change":"2024-04-25T10:52:32.463452+0000","last_active":"2024-04-25T10:52:34.154852+0000","last_peered":"2024-04-25T10:52:34.154852+0000","last_clean":"2024-04-25T10:52:34.154852+0000","last_became_active":"2024-04-25T10:52:32.461668+0000","last_became_peered":"2024-04-25T10:52:32.461668+0000","last_unstale":"2024-04-25T10:52:34.154852+0000","last_undegraded":"2024-04-25T10:52:34.154852+0000","last_fullsized":"2024-04-25T10:52:34.154852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.871762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456702+0000","last_change":"2024-04-25T10:52:31.698507+0000","last_active":"2024-04-25T10:52:33.456702+0000","last_peered":"2024-04-25T10:52:33.456702+0000","last_clean":"2024-04-25T10:52:33.456702+0000","last_became_active":"2024-04-25T10:52:31.698148+0000","last_became_peered":"2024-04-25T10:52:31.698148+0000","last_unstale":"2024-04-25T10:52:33.456702+0000","last_undegraded":"2024-04-25T10:52:33.456702+0000","last_fullsized":"2024-04-25T10:52:33.456702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:48.993234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154021+0000","last_change":"2024-04-25T10:52:32.465874+0000","last_active":"2024-04-25T10:52:34.154021+0000","last_peered":"2024-04-25T10:52:34.154021+0000","last_clean":"2024-04-25T10:52:34.154021+0000","last_became_active":"2024-04-25T10:52:32.465511+0000","last_became_peered":"2024-04-25T10:52:32.465511+0000","last_unstale":"2024-04-25T10:52:34.154021+0000","last_undegraded":"2024-04-25T10:52:34.154021+0000","last_fullsized":"2024-04-25T10:52:34.154021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:15.274909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455263+0000","last_change":"2024-04-25T10:52:31.691074+0000","last_active":"2024-04-25T10:52:33.455263+0000","last_peered":"2024-04-25T10:52:33.455263+0000","last_clean":"2024-04-25T10:52:33.455263+0000","last_became_active":"2024-04-25T10:52:31.690402+0000","last_became_peered":"2024-04-25T10:52:31.690402+0000","last_unstale":"2024-04-25T10:52:33.455263+0000","last_undegraded":"2024-04-25T10:52:33.455263+0000","last_fullsized":"2024-04-25T10:52:33.455263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.529860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154509+0000","last_change":"2024-04-25T10:52:32.477968+0000","last_active":"2024-04-25T10:52:34.154509+0000","last_peered":"2024-04-25T10:52:34.154509+0000","last_clean":"2024-04-25T10:52:34.154509+0000","last_became_active":"2024-04-25T10:52:32.476395+0000","last_became_peered":"2024-04-25T10:52:32.476395+0000","last_unstale":"2024-04-25T10:52:34.154509+0000","last_undegraded":"2024-04-25T10:52:34.154509+0000","last_fullsized":"2024-04-25T10:52:34.154509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:13.128910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154891+0000","last_change":"2024-04-25T10:52:31.698934+0000","last_active":"2024-04-25T10:52:34.154891+0000","last_peered":"2024-04-25T10:52:34.154891+0000","last_clean":"2024-04-25T10:52:34.154891+0000","last_became_active":"2024-04-25T10:52:31.698565+0000","last_became_peered":"2024-04-25T10:52:31.698565+0000","last_unstale":"2024-04-25T10:52:34.154891+0000","last_undegraded":"2024-04-25T10:52:34.154891+0000","last_fullsized":"2024-04-25T10:52:34.154891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.360617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154004+0000","last_change":"2024-04-25T10:52:32.477538+0000","last_active":"2024-04-25T10:52:34.154004+0000","last_peered":"2024-04-25T10:52:34.154004+0000","last_clean":"2024-04-25T10:52:34.154004+0000","last_became_active":"2024-04-25T10:52:32.476088+0000","last_became_peered":"2024-04-25T10:52:32.476088+0000","last_unstale":"2024-04-25T10:52:34.154004+0000","last_undegraded":"2024-04-25T10:52:34.154004+0000","last_fullsized":"2024-04-25T10:52:34.154004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.987071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154129+0000","last_change":"2024-04-25T10:52:32.469335+0000","last_active":"2024-04-25T10:52:34.154129+0000","last_peered":"2024-04-25T10:52:34.154129+0000","last_clean":"2024-04-25T10:52:34.154129+0000","last_became_active":"2024-04-25T10:52:32.468921+0000","last_became_peered":"2024-04-25T10:52:32.468921+0000","last_unstale":"2024-04-25T10:52:34.154129+0000","last_undegraded":"2024-04-25T10:52:34.154129+0000","last_fullsized":"2024-04-25T10:52:34.154129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:26.499243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791540+0000","last_change":"2024-04-25T10:52:31.706700+0000","last_active":"2024-04-25T10:52:33.791540+0000","last_peered":"2024-04-25T10:52:33.791540+0000","last_clean":"2024-04-25T10:52:33.791540+0000","last_became_active":"2024-04-25T10:52:31.706392+0000","last_became_peered":"2024-04-25T10:52:31.706392+0000","last_unstale":"2024-04-25T10:52:33.791540+0000","last_undegraded":"2024-04-25T10:52:33.791540+0000","last_fullsized":"2024-04-25T10:52:33.791540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:58.867351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715326+0000","last_change":"2024-04-25T10:52:32.462603+0000","last_active":"2024-04-25T10:52:33.715326+0000","last_peered":"2024-04-25T10:52:33.715326+0000","last_clean":"2024-04-25T10:52:33.715326+0000","last_became_active":"2024-04-25T10:52:32.462238+0000","last_became_peered":"2024-04-25T10:52:32.462238+0000","last_unstale":"2024-04-25T10:52:33.715326+0000","last_undegraded":"2024-04-25T10:52:33.715326+0000","last_fullsized":"2024-04-25T10:52:33.715326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.555683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456322+0000","last_change":"2024-04-25T10:52:31.694054+0000","last_active":"2024-04-25T10:52:33.456322+0000","last_peered":"2024-04-25T10:52:33.456322+0000","last_clean":"2024-04-25T10:52:33.456322+0000","last_became_active":"2024-04-25T10:52:31.693693+0000","last_became_peered":"2024-04-25T10:52:31.693693+0000","last_unstale":"2024-04-25T10:52:33.456322+0000","last_undegraded":"2024-04-25T10:52:33.456322+0000","last_fullsized":"2024-04-25T10:52:33.456322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:22.238131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154745+0000","last_change":"2024-04-25T10:51:22.730925+0000","last_active":"2024-04-25T10:52:34.154745+0000","last_peered":"2024-04-25T10:52:34.154745+0000","last_clean":"2024-04-25T10:52:34.154745+0000","last_became_active":"2024-04-25T10:51:22.725168+0000","last_became_peered":"2024-04-25T10:51:22.725168+0000","last_unstale":"2024-04-25T10:52:34.154745+0000","last_undegraded":"2024-04-25T10:52:34.154745+0000","last_fullsized":"2024-04-25T10:52:34.154745+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_clean_scrub_stamp":"2024-04-25T10:47:56.758246+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:02.416333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791576+0000","last_change":"2024-04-25T10:52:32.461608+0000","last_active":"2024-04-25T10:52:33.791576+0000","last_peered":"2024-04-25T10:52:33.791576+0000","last_clean":"2024-04-25T10:52:33.791576+0000","last_became_active":"2024-04-25T10:52:32.461210+0000","last_became_peered":"2024-04-25T10:52:32.461210+0000","last_unstale":"2024-04-25T10:52:33.791576+0000","last_undegraded":"2024-04-25T10:52:33.791576+0000","last_fullsized":"2024-04-25T10:52:33.791576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:14.897053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154935+0000","last_change":"2024-04-25T10:52:31.701003+0000","last_active":"2024-04-25T10:52:34.154935+0000","last_peered":"2024-04-25T10:52:34.154935+0000","last_clean":"2024-04-25T10:52:34.154935+0000","last_became_active":"2024-04-25T10:52:31.699623+0000","last_became_peered":"2024-04-25T10:52:31.699623+0000","last_unstale":"2024-04-25T10:52:34.154935+0000","last_undegraded":"2024-04-25T10:52:34.154935+0000","last_fullsized":"2024-04-25T10:52:34.154935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:45.175760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456723+0000","last_change":"2024-04-25T10:52:31.702595+0000","last_active":"2024-04-25T10:52:33.456723+0000","last_peered":"2024-04-25T10:52:33.456723+0000","last_clean":"2024-04-25T10:52:33.456723+0000","last_became_active":"2024-04-25T10:52:31.702124+0000","last_became_peered":"2024-04-25T10:52:31.702124+0000","last_unstale":"2024-04-25T10:52:33.456723+0000","last_undegraded":"2024-04-25T10:52:33.456723+0000","last_fullsized":"2024-04-25T10:52:33.456723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:54.351859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.792032+0000","last_change":"2024-04-25T10:52:32.467987+0000","last_active":"2024-04-25T10:52:33.792032+0000","last_peered":"2024-04-25T10:52:33.792032+0000","last_clean":"2024-04-25T10:52:33.792032+0000","last_became_active":"2024-04-25T10:52:32.467613+0000","last_became_peered":"2024-04-25T10:52:32.467613+0000","last_unstale":"2024-04-25T10:52:33.792032+0000","last_undegraded":"2024-04-25T10:52:33.792032+0000","last_fullsized":"2024-04-25T10:52:33.792032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:00.112102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.713033+0000","last_change":"2024-04-25T10:52:31.699868+0000","last_active":"2024-04-25T10:52:33.713033+0000","last_peered":"2024-04-25T10:52:33.713033+0000","last_clean":"2024-04-25T10:52:33.713033+0000","last_became_active":"2024-04-25T10:52:31.699470+0000","last_became_peered":"2024-04-25T10:52:31.699470+0000","last_unstale":"2024-04-25T10:52:33.713033+0000","last_undegraded":"2024-04-25T10:52:33.713033+0000","last_fullsized":"2024-04-25T10:52:33.713033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.161821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715642+0000","last_change":"2024-04-25T10:52:32.459082+0000","last_active":"2024-04-25T10:52:33.715642+0000","last_peered":"2024-04-25T10:52:33.715642+0000","last_clean":"2024-04-25T10:52:33.715642+0000","last_became_active":"2024-04-25T10:52:32.458190+0000","last_became_peered":"2024-04-25T10:52:32.458190+0000","last_unstale":"2024-04-25T10:52:33.715642+0000","last_undegraded":"2024-04-25T10:52:33.715642+0000","last_fullsized":"2024-04-25T10:52:33.715642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.280762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456674+0000","last_change":"2024-04-25T10:52:31.700636+0000","last_active":"2024-04-25T10:52:33.456674+0000","last_peered":"2024-04-25T10:52:33.456674+0000","last_clean":"2024-04-25T10:52:33.456674+0000","last_became_active":"2024-04-25T10:52:31.700268+0000","last_became_peered":"2024-04-25T10:52:31.700268+0000","last_unstale":"2024-04-25T10:52:33.456674+0000","last_undegraded":"2024-04-25T10:52:33.456674+0000","last_fullsized":"2024-04-25T10:52:33.456674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.439491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348914+0000","last_change":"2024-04-25T10:52:32.467406+0000","last_active":"2024-04-25T10:52:34.348914+0000","last_peered":"2024-04-25T10:52:34.348914+0000","last_clean":"2024-04-25T10:52:34.348914+0000","last_became_active":"2024-04-25T10:52:32.467053+0000","last_became_peered":"2024-04-25T10:52:32.467053+0000","last_unstale":"2024-04-25T10:52:34.348914+0000","last_undegraded":"2024-04-25T10:52:34.348914+0000","last_fullsized":"2024-04-25T10:52:34.348914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153258+0000","last_change":"2024-04-25T10:52:31.689623+0000","last_active":"2024-04-25T10:52:34.153258+0000","last_peered":"2024-04-25T10:52:34.153258+0000","last_clean":"2024-04-25T10:52:34.153258+0000","last_became_active":"2024-04-25T10:52:31.689261+0000","last_became_peered":"2024-04-25T10:52:31.689261+0000","last_unstale":"2024-04-25T10:52:34.153258+0000","last_undegraded":"2024-04-25T10:52:34.153258+0000","last_fullsized":"2024-04-25T10:52:34.153258+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:42.787964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456293+0000","last_change":"2024-04-25T10:52:32.472709+0000","last_active":"2024-04-25T10:52:33.456293+0000","last_peered":"2024-04-25T10:52:33.456293+0000","last_clean":"2024-04-25T10:52:33.456293+0000","last_became_active":"2024-04-25T10:52:32.472331+0000","last_became_peered":"2024-04-25T10:52:32.472331+0000","last_unstale":"2024-04-25T10:52:33.456293+0000","last_undegraded":"2024-04-25T10:52:33.456293+0000","last_fullsized":"2024-04-25T10:52:33.456293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.047682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715149+0000","last_change":"2024-04-25T10:52:31.697360+0000","last_active":"2024-04-25T10:52:33.715149+0000","last_peered":"2024-04-25T10:52:33.715149+0000","last_clean":"2024-04-25T10:52:33.715149+0000","last_became_active":"2024-04-25T10:52:31.696991+0000","last_became_peered":"2024-04-25T10:52:31.696991+0000","last_unstale":"2024-04-25T10:52:33.715149+0000","last_undegraded":"2024-04-25T10:52:33.715149+0000","last_fullsized":"2024-04-25T10:52:33.715149+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:38.395372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153990+0000","last_change":"2024-04-25T10:52:32.472505+0000","last_active":"2024-04-25T10:52:34.153990+0000","last_peered":"2024-04-25T10:52:34.153990+0000","last_clean":"2024-04-25T10:52:34.153990+0000","last_became_active":"2024-04-25T10:52:32.472113+0000","last_became_peered":"2024-04-25T10:52:32.472113+0000","last_unstale":"2024-04-25T10:52:34.153990+0000","last_undegraded":"2024-04-25T10:52:34.153990+0000","last_fullsized":"2024-04-25T10:52:34.153990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:20.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154274+0000","last_change":"2024-04-25T10:52:31.696488+0000","last_active":"2024-04-25T10:52:34.154274+0000","last_peered":"2024-04-25T10:52:34.154274+0000","last_clean":"2024-04-25T10:52:34.154274+0000","last_became_active":"2024-04-25T10:52:31.696062+0000","last_became_peered":"2024-04-25T10:52:31.696062+0000","last_unstale":"2024-04-25T10:52:34.154274+0000","last_undegraded":"2024-04-25T10:52:34.154274+0000","last_fullsized":"2024-04-25T10:52:34.154274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:24.329938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715801+0000","last_change":"2024-04-25T10:52:32.474555+0000","last_active":"2024-04-25T10:52:33.715801+0000","last_peered":"2024-04-25T10:52:33.715801+0000","last_clean":"2024-04-25T10:52:33.715801+0000","last_became_active":"2024-04-25T10:52:32.474001+0000","last_became_peered":"2024-04-25T10:52:32.474001+0000","last_unstale":"2024-04-25T10:52:33.715801+0000","last_undegraded":"2024-04-25T10:52:33.715801+0000","last_fullsized":"2024-04-25T10:52:33.715801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.775966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'2","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456431+0000","last_change":"2024-04-25T10:52:31.702251+0000","last_active":"2024-04-25T10:52:33.456431+0000","last_peered":"2024-04-25T10:52:33.456431+0000","last_clean":"2024-04-25T10:52:33.456431+0000","last_became_active":"2024-04-25T10:52:31.701763+0000","last_became_peered":"2024-04-25T10:52:31.701763+0000","last_unstale":"2024-04-25T10:52:33.456431+0000","last_undegraded":"2024-04-25T10:52:33.456431+0000","last_fullsized":"2024-04-25T10:52:33.456431+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155330+0000","last_change":"2024-04-25T10:52:32.465494+0000","last_active":"2024-04-25T10:52:34.155330+0000","last_peered":"2024-04-25T10:52:34.155330+0000","last_clean":"2024-04-25T10:52:34.155330+0000","last_became_active":"2024-04-25T10:52:32.463855+0000","last_became_peered":"2024-04-25T10:52:32.463855+0000","last_unstale":"2024-04-25T10:52:34.155330+0000","last_undegraded":"2024-04-25T10:52:34.155330+0000","last_fullsized":"2024-04-25T10:52:34.155330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:40.970580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155104+0000","last_change":"2024-04-25T10:52:31.700749+0000","last_active":"2024-04-25T10:52:34.155104+0000","last_peered":"2024-04-25T10:52:34.155104+0000","last_clean":"2024-04-25T10:52:34.155104+0000","last_became_active":"2024-04-25T10:52:31.699109+0000","last_became_peered":"2024-04-25T10:52:31.699109+0000","last_unstale":"2024-04-25T10:52:34.155104+0000","last_undegraded":"2024-04-25T10:52:34.155104+0000","last_fullsized":"2024-04-25T10:52:34.155104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:48.405333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155314+0000","last_change":"2024-04-25T10:52:32.468292+0000","last_active":"2024-04-25T10:52:34.155314+0000","last_peered":"2024-04-25T10:52:34.155314+0000","last_clean":"2024-04-25T10:52:34.155314+0000","last_became_active":"2024-04-25T10:52:32.467934+0000","last_became_peered":"2024-04-25T10:52:32.467934+0000","last_unstale":"2024-04-25T10:52:34.155314+0000","last_undegraded":"2024-04-25T10:52:34.155314+0000","last_fullsized":"2024-04-25T10:52:34.155314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.366735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154543+0000","last_change":"2024-04-25T10:52:31.705025+0000","last_active":"2024-04-25T10:52:34.154543+0000","last_peered":"2024-04-25T10:52:34.154543+0000","last_clean":"2024-04-25T10:52:34.154543+0000","last_became_active":"2024-04-25T10:52:31.704532+0000","last_became_peered":"2024-04-25T10:52:31.704532+0000","last_unstale":"2024-04-25T10:52:34.154543+0000","last_undegraded":"2024-04-25T10:52:34.154543+0000","last_fullsized":"2024-04-25T10:52:34.154543+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.371778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155155+0000","last_change":"2024-04-25T10:52:32.464149+0000","last_active":"2024-04-25T10:52:34.155155+0000","last_peered":"2024-04-25T10:52:34.155155+0000","last_clean":"2024-04-25T10:52:34.155155+0000","last_became_active":"2024-04-25T10:52:32.463630+0000","last_became_peered":"2024-04-25T10:52:32.463630+0000","last_unstale":"2024-04-25T10:52:34.155155+0000","last_undegraded":"2024-04-25T10:52:34.155155+0000","last_fullsized":"2024-04-25T10:52:34.155155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:38.219914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791752+0000","last_change":"2024-04-25T10:52:31.705770+0000","last_active":"2024-04-25T10:52:33.791752+0000","last_peered":"2024-04-25T10:52:33.791752+0000","last_clean":"2024-04-25T10:52:33.791752+0000","last_became_active":"2024-04-25T10:52:31.705391+0000","last_became_peered":"2024-04-25T10:52:31.705391+0000","last_unstale":"2024-04-25T10:52:33.791752+0000","last_undegraded":"2024-04-25T10:52:33.791752+0000","last_fullsized":"2024-04-25T10:52:33.791752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:32.831435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155504+0000","last_change":"2024-04-25T10:52:32.465550+0000","last_active":"2024-04-25T10:52:34.155504+0000","last_peered":"2024-04-25T10:52:34.155504+0000","last_clean":"2024-04-25T10:52:34.155504+0000","last_became_active":"2024-04-25T10:52:32.465034+0000","last_became_peered":"2024-04-25T10:52:32.465034+0000","last_unstale":"2024-04-25T10:52:34.155504+0000","last_undegraded":"2024-04-25T10:52:34.155504+0000","last_fullsized":"2024-04-25T10:52:34.155504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.810469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154283+0000","last_change":"2024-04-25T10:52:32.470812+0000","last_active":"2024-04-25T10:52:34.154283+0000","last_peered":"2024-04-25T10:52:34.154283+0000","last_clean":"2024-04-25T10:52:34.154283+0000","last_became_active":"2024-04-25T10:52:32.469620+0000","last_became_peered":"2024-04-25T10:52:32.469620+0000","last_unstale":"2024-04-25T10:52:34.154283+0000","last_undegraded":"2024-04-25T10:52:34.154283+0000","last_fullsized":"2024-04-25T10:52:34.154283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154471+0000","last_change":"2024-04-25T10:52:32.464762+0000","last_active":"2024-04-25T10:52:34.154471+0000","last_peered":"2024-04-25T10:52:34.154471+0000","last_clean":"2024-04-25T10:52:34.154471+0000","last_became_active":"2024-04-25T10:52:32.464316+0000","last_became_peered":"2024-04-25T10:52:32.464316+0000","last_unstale":"2024-04-25T10:52:34.154471+0000","last_undegraded":"2024-04-25T10:52:34.154471+0000","last_fullsized":"2024-04-25T10:52:34.154471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:55.526572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154197+0000","last_change":"2024-04-25T10:52:32.466429+0000","last_active":"2024-04-25T10:52:34.154197+0000","last_peered":"2024-04-25T10:52:34.154197+0000","last_clean":"2024-04-25T10:52:34.154197+0000","last_became_active":"2024-04-25T10:52:32.465132+0000","last_became_peered":"2024-04-25T10:52:32.465132+0000","last_unstale":"2024-04-25T10:52:34.154197+0000","last_undegraded":"2024-04-25T10:52:34.154197+0000","last_fullsized":"2024-04-25T10:52:34.154197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:14.274441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348595+0000","last_change":"2024-04-25T10:52:32.464666+0000","last_active":"2024-04-25T10:52:34.348595+0000","last_peered":"2024-04-25T10:52:34.348595+0000","last_clean":"2024-04-25T10:52:34.348595+0000","last_became_active":"2024-04-25T10:52:32.464226+0000","last_became_peered":"2024-04-25T10:52:32.464226+0000","last_unstale":"2024-04-25T10:52:34.348595+0000","last_undegraded":"2024-04-25T10:52:34.348595+0000","last_fullsized":"2024-04-25T10:52:34.348595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:39.388542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791932+0000","last_change":"2024-04-25T10:52:32.465199+0000","last_active":"2024-04-25T10:52:33.791932+0000","last_peered":"2024-04-25T10:52:33.791932+0000","last_clean":"2024-04-25T10:52:33.791932+0000","last_became_active":"2024-04-25T10:52:32.464285+0000","last_became_peered":"2024-04-25T10:52:32.464285+0000","last_unstale":"2024-04-25T10:52:33.791932+0000","last_undegraded":"2024-04-25T10:52:33.791932+0000","last_fullsized":"2024-04-25T10:52:33.791932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.010994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154224+0000","last_change":"2024-04-25T10:52:32.469140+0000","last_active":"2024-04-25T10:52:34.154224+0000","last_peered":"2024-04-25T10:52:34.154224+0000","last_clean":"2024-04-25T10:52:34.154224+0000","last_became_active":"2024-04-25T10:52:32.468344+0000","last_became_peered":"2024-04-25T10:52:32.468344+0000","last_unstale":"2024-04-25T10:52:34.154224+0000","last_undegraded":"2024-04-25T10:52:34.154224+0000","last_fullsized":"2024-04-25T10:52:34.154224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:07.228059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155386+0000","last_change":"2024-04-25T10:52:32.469326+0000","last_active":"2024-04-25T10:52:34.155386+0000","last_peered":"2024-04-25T10:52:34.155386+0000","last_clean":"2024-04-25T10:52:34.155386+0000","last_became_active":"2024-04-25T10:52:32.468905+0000","last_became_peered":"2024-04-25T10:52:32.468905+0000","last_unstale":"2024-04-25T10:52:34.155386+0000","last_undegraded":"2024-04-25T10:52:34.155386+0000","last_fullsized":"2024-04-25T10:52:34.155386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:31.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715511+0000","last_change":"2024-04-25T10:52:32.459391+0000","last_active":"2024-04-25T10:52:33.715511+0000","last_peered":"2024-04-25T10:52:33.715511+0000","last_clean":"2024-04-25T10:52:33.715511+0000","last_became_active":"2024-04-25T10:52:32.458702+0000","last_became_peered":"2024-04-25T10:52:32.458702+0000","last_unstale":"2024-04-25T10:52:33.715511+0000","last_undegraded":"2024-04-25T10:52:33.715511+0000","last_fullsized":"2024-04-25T10:52:33.715511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.475714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153957+0000","last_change":"2024-04-25T10:52:32.467395+0000","last_active":"2024-04-25T10:52:34.153957+0000","last_peered":"2024-04-25T10:52:34.153957+0000","last_clean":"2024-04-25T10:52:34.153957+0000","last_became_active":"2024-04-25T10:52:32.466976+0000","last_became_peered":"2024-04-25T10:52:32.466976+0000","last_unstale":"2024-04-25T10:52:34.153957+0000","last_undegraded":"2024-04-25T10:52:34.153957+0000","last_fullsized":"2024-04-25T10:52:34.153957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:22.011064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715052+0000","last_change":"2024-04-25T10:52:32.458874+0000","last_active":"2024-04-25T10:52:33.715052+0000","last_peered":"2024-04-25T10:52:33.715052+0000","last_clean":"2024-04-25T10:52:33.715052+0000","last_became_active":"2024-04-25T10:52:32.458517+0000","last_became_peered":"2024-04-25T10:52:32.458517+0000","last_unstale":"2024-04-25T10:52:33.715052+0000","last_undegraded":"2024-04-25T10:52:33.715052+0000","last_fullsized":"2024-04-25T10:52:33.715052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:12.634521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153846+0000","last_change":"2024-04-25T10:52:32.473068+0000","last_active":"2024-04-25T10:52:34.153846+0000","last_peered":"2024-04-25T10:52:34.153846+0000","last_clean":"2024-04-25T10:52:34.153846+0000","last_became_active":"2024-04-25T10:52:32.472713+0000","last_became_peered":"2024-04-25T10:52:32.472713+0000","last_unstale":"2024-04-25T10:52:34.153846+0000","last_undegraded":"2024-04-25T10:52:34.153846+0000","last_fullsized":"2024-04-25T10:52:34.153846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:43.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791929+0000","last_change":"2024-04-25T10:52:32.468492+0000","last_active":"2024-04-25T10:52:33.791929+0000","last_peered":"2024-04-25T10:52:33.791929+0000","last_clean":"2024-04-25T10:52:33.791929+0000","last_became_active":"2024-04-25T10:52:32.468155+0000","last_became_peered":"2024-04-25T10:52:32.468155+0000","last_unstale":"2024-04-25T10:52:33.791929+0000","last_undegraded":"2024-04-25T10:52:33.791929+0000","last_fullsized":"2024-04-25T10:52:33.791929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.934230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154824+0000","last_change":"2024-04-25T10:52:32.463057+0000","last_active":"2024-04-25T10:52:34.154824+0000","last_peered":"2024-04-25T10:52:34.154824+0000","last_clean":"2024-04-25T10:52:34.154824+0000","last_became_active":"2024-04-25T10:52:32.460983+0000","last_became_peered":"2024-04-25T10:52:32.460983+0000","last_unstale":"2024-04-25T10:52:34.154824+0000","last_undegraded":"2024-04-25T10:52:34.154824+0000","last_fullsized":"2024-04-25T10:52:34.154824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":7230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645325,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":819200,"data_stored":394047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841554,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712568,"statfs":{"total":95995035648,"available":95961669632,"internally_reserved":0,"allocated":1437696,"data_stored":986221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":394629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":827392,"data_stored":394651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1417216,"data_stored":984997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593767,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":819200,"data_stored":394047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":831488,"data_stored":395659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":394137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":395693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":827392,"data_stored":394151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":823296,"data_stored":394069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789996,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1425408,"data_stored":984597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:52:39.678 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:52:39.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 -- 172.21.15.19:0/3025212646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 msgr2=0x7f6cb81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3025212646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb81036c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009900 tx=0x7f6ca802a410 comp rx=0 tx=0).stop 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 -- 172.21.15.19:0/3025212646 shutdown_connections 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3025212646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb8108900 0x7f6cb810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3025212646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8103c00 0x7f6cb8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3025212646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb81036c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 -- 172.21.15.19:0/3025212646 >> 172.21.15.19:0/3025212646 conn(0x7f6cb80fdbd0 msgr2=0x7f6cb8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 -- 172.21.15.19:0/3025212646 shutdown_connections 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.826+0000 7f6cbe772640 1 -- 172.21.15.19:0/3025212646 wait complete. 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 Processor -- start 2024-04-25T10:52:39.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 -- start start 2024-04-25T10:52:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8103c00 0x7f6cb819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb8108900 0x7f6cb81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb810e570 con 0x7f6cb81029c0 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb810e3f0 con 0x7f6cb8103c00 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbe772640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb810e6f0 con 0x7f6cb8108900 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41352/0 (socket says 172.21.15.19:41352) 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cb7fff640 1 -- 172.21.15.19:0/3913795459 learned_addr learned my addr 172.21.15.19:0/3913795459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.827+0000 7f6cbcd0f640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb8108900 0x7f6cb81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb7fff640 1 -- 172.21.15.19:0/3913795459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb8108900 msgr2=0x7f6cb81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb7fff640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb8108900 0x7f6cb81a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb77fe640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8103c00 0x7f6cb819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb7fff640 1 -- 172.21.15.19:0/3913795459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8103c00 msgr2=0x7f6cb819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb7fff640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8103c00 0x7f6cb819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb7fff640 1 -- 172.21.15.19:0/3913795459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb81a1dd0 con 0x7f6cb81029c0 2024-04-25T10:52:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb7fff640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb819a290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8031440 tx=0x7f6ca8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb57fa640 1 -- 172.21.15.19:0/3913795459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca803f070 con 0x7f6cb81029c0 2024-04-25T10:52:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb57fa640 1 -- 172.21.15.19:0/3913795459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca803b460 con 0x7f6cb81029c0 2024-04-25T10:52:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb81a2060 con 0x7f6cb81029c0 2024-04-25T10:52:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cb57fa640 1 -- 172.21.15.19:0/3913795459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8035030 con 0x7f6cb81029c0 2024-04-25T10:52:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.828+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb81a25d0 con 0x7f6cb81029c0 2024-04-25T10:52:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.829+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb81a2830 con 0x7f6cb81029c0 2024-04-25T10:52:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.830+0000 7f6cb57fa640 1 -- 172.21.15.19:0/3913795459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6ca8042070 con 0x7f6cb81029c0 2024-04-25T10:52:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.831+0000 7f6cb57fa640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6ca00afb80 0x7f6ca00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.831+0000 7f6cb57fa640 1 -- 172.21.15.19:0/3913795459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f6ca80ff5d0 con 0x7f6cb81029c0 2024-04-25T10:52:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.831+0000 7f6cb77fe640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6ca00afb80 0x7f6ca00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.832+0000 7f6cb77fe640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6ca00afb80 0x7f6ca00b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6cac005fd0 tx=0x7f6cac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:39.836+0000 7f6cb57fa640 1 -- 172.21.15.19:0/3913795459 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ca80c8200 con 0x7f6cb81029c0 2024-04-25T10:52:40.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.034+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6cb810a500 con 0x7f6ca00afb80 2024-04-25T10:52:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.041+0000 7f6cb57fa640 1 -- 172.21.15.19:0/3913795459 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258689 (secure 0 0 0) 0x7f6cb810a500 con 0x7f6ca00afb80 2024-04-25T10:52:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.044+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6ca00afb80 msgr2=0x7f6ca00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.044+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6ca00afb80 0x7f6ca00b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6cac005fd0 tx=0x7f6cac005850 comp rx=0 tx=0).stop 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.044+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 msgr2=0x7f6cb819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.044+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb819a290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8031440 tx=0x7f6ca8003d50 comp rx=0 tx=0).stop 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.044+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 shutdown_connections 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.045+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb8108900 0x7f6cb81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.045+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8103c00 0x7f6cb819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.045+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6ca00afb80 0x7f6ca00b2090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.045+0000 7f6cbe772640 1 --2- 172.21.15.19:0/3913795459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb81029c0 0x7f6cb819a290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.045+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 >> 172.21.15.19:0/3913795459 conn(0x7f6cb80fdbd0 msgr2=0x7f6cb8109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.045+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 shutdown_connections 2024-04-25T10:52:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:40.045+0000 7f6cbe772640 1 -- 172.21.15.19:0/3913795459 wait complete. 2024-04-25T10:52:40.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:39 smithi019 ceph-mon[29209]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:39 smithi019 ceph-mon[29209]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 811 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T10:52:40.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:39 smithi019 ceph-mon[29209]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:39 smithi019 ceph-mon[29209]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:39 smithi019 ceph-mon[29209]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T10:52:40.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:39 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1403023294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:40.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:39 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/257813967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:40.057 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-04-25T10:52:39.994271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":86,"num_read_kb":74,"num_write":168,"num_write_kb":2164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364840,"kb_used_data":11436,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576984,"statfs":{"total":1151940427776,"available":1151566831616,"internally_reserved":0,"allocated":11710464,"data_stored":6506898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001931"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154931+0000","last_change":"2024-04-25T10:52:32.474738+0000","last_active":"2024-04-25T10:52:34.154931+0000","last_peered":"2024-04-25T10:52:34.154931+0000","last_clean":"2024-04-25T10:52:34.154931+0000","last_became_active":"2024-04-25T10:52:32.473812+0000","last_became_peered":"2024-04-25T10:52:32.473812+0000","last_unstale":"2024-04-25T10:52:34.154931+0000","last_undegraded":"2024-04-25T10:52:34.154931+0000","last_fullsized":"2024-04-25T10:52:34.154931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:25.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348904+0000","last_change":"2024-04-25T10:52:32.475491+0000","last_active":"2024-04-25T10:52:34.348904+0000","last_peered":"2024-04-25T10:52:34.348904+0000","last_clean":"2024-04-25T10:52:34.348904+0000","last_became_active":"2024-04-25T10:52:32.474164+0000","last_became_peered":"2024-04-25T10:52:32.474164+0000","last_unstale":"2024-04-25T10:52:34.348904+0000","last_undegraded":"2024-04-25T10:52:34.348904+0000","last_fullsized":"2024-04-25T10:52:34.348904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:58.838292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348730+0000","last_change":"2024-04-25T10:52:32.471349+0000","last_active":"2024-04-25T10:52:34.348730+0000","last_peered":"2024-04-25T10:52:34.348730+0000","last_clean":"2024-04-25T10:52:34.348730+0000","last_became_active":"2024-04-25T10:52:32.470972+0000","last_became_peered":"2024-04-25T10:52:32.470972+0000","last_unstale":"2024-04-25T10:52:34.348730+0000","last_undegraded":"2024-04-25T10:52:34.348730+0000","last_fullsized":"2024-04-25T10:52:34.348730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:28.634537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456431+0000","last_change":"2024-04-25T10:52:32.470212+0000","last_active":"2024-04-25T10:52:33.456431+0000","last_peered":"2024-04-25T10:52:33.456431+0000","last_clean":"2024-04-25T10:52:33.456431+0000","last_became_active":"2024-04-25T10:52:32.469642+0000","last_became_peered":"2024-04-25T10:52:32.469642+0000","last_unstale":"2024-04-25T10:52:33.456431+0000","last_undegraded":"2024-04-25T10:52:33.456431+0000","last_fullsized":"2024-04-25T10:52:33.456431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:08.231679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456367+0000","last_change":"2024-04-25T10:52:32.456568+0000","last_active":"2024-04-25T10:52:33.456367+0000","last_peered":"2024-04-25T10:52:33.456367+0000","last_clean":"2024-04-25T10:52:33.456367+0000","last_became_active":"2024-04-25T10:52:32.456051+0000","last_became_peered":"2024-04-25T10:52:32.456051+0000","last_unstale":"2024-04-25T10:52:33.456367+0000","last_undegraded":"2024-04-25T10:52:33.456367+0000","last_fullsized":"2024-04-25T10:52:33.456367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154170+0000","last_change":"2024-04-25T10:52:32.479781+0000","last_active":"2024-04-25T10:52:34.154170+0000","last_peered":"2024-04-25T10:52:34.154170+0000","last_clean":"2024-04-25T10:52:34.154170+0000","last_became_active":"2024-04-25T10:52:32.479213+0000","last_became_peered":"2024-04-25T10:52:32.479213+0000","last_unstale":"2024-04-25T10:52:34.154170+0000","last_undegraded":"2024-04-25T10:52:34.154170+0000","last_fullsized":"2024-04-25T10:52:34.154170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:16.239969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154843+0000","last_change":"2024-04-25T10:52:32.471430+0000","last_active":"2024-04-25T10:52:34.154843+0000","last_peered":"2024-04-25T10:52:34.154843+0000","last_clean":"2024-04-25T10:52:34.154843+0000","last_became_active":"2024-04-25T10:52:32.470518+0000","last_became_peered":"2024-04-25T10:52:32.470518+0000","last_unstale":"2024-04-25T10:52:34.154843+0000","last_undegraded":"2024-04-25T10:52:34.154843+0000","last_fullsized":"2024-04-25T10:52:34.154843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:12.846174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.714871+0000","last_change":"2024-04-25T10:52:32.472545+0000","last_active":"2024-04-25T10:52:33.714871+0000","last_peered":"2024-04-25T10:52:33.714871+0000","last_clean":"2024-04-25T10:52:33.714871+0000","last_became_active":"2024-04-25T10:52:32.472151+0000","last_became_peered":"2024-04-25T10:52:32.472151+0000","last_unstale":"2024-04-25T10:52:33.714871+0000","last_undegraded":"2024-04-25T10:52:33.714871+0000","last_fullsized":"2024-04-25T10:52:33.714871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:36.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154239+0000","last_change":"2024-04-25T10:52:32.466223+0000","last_active":"2024-04-25T10:52:34.154239+0000","last_peered":"2024-04-25T10:52:34.154239+0000","last_clean":"2024-04-25T10:52:34.154239+0000","last_became_active":"2024-04-25T10:52:32.465498+0000","last_became_peered":"2024-04-25T10:52:32.465498+0000","last_unstale":"2024-04-25T10:52:34.154239+0000","last_undegraded":"2024-04-25T10:52:34.154239+0000","last_fullsized":"2024-04-25T10:52:34.154239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:25.924450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154635+0000","last_change":"2024-04-25T10:52:32.470601+0000","last_active":"2024-04-25T10:52:34.154635+0000","last_peered":"2024-04-25T10:52:34.154635+0000","last_clean":"2024-04-25T10:52:34.154635+0000","last_became_active":"2024-04-25T10:52:32.468984+0000","last_became_peered":"2024-04-25T10:52:32.468984+0000","last_unstale":"2024-04-25T10:52:34.154635+0000","last_undegraded":"2024-04-25T10:52:34.154635+0000","last_fullsized":"2024-04-25T10:52:34.154635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155296+0000","last_change":"2024-04-25T10:52:32.463907+0000","last_active":"2024-04-25T10:52:34.155296+0000","last_peered":"2024-04-25T10:52:34.155296+0000","last_clean":"2024-04-25T10:52:34.155296+0000","last_became_active":"2024-04-25T10:52:32.463536+0000","last_became_peered":"2024-04-25T10:52:32.463536+0000","last_unstale":"2024-04-25T10:52:34.155296+0000","last_undegraded":"2024-04-25T10:52:34.155296+0000","last_fullsized":"2024-04-25T10:52:34.155296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.503805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456424+0000","last_change":"2024-04-25T10:52:32.473646+0000","last_active":"2024-04-25T10:52:33.456424+0000","last_peered":"2024-04-25T10:52:33.456424+0000","last_clean":"2024-04-25T10:52:33.456424+0000","last_became_active":"2024-04-25T10:52:32.473310+0000","last_became_peered":"2024-04-25T10:52:32.473310+0000","last_unstale":"2024-04-25T10:52:33.456424+0000","last_undegraded":"2024-04-25T10:52:33.456424+0000","last_fullsized":"2024-04-25T10:52:33.456424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.691293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791849+0000","last_change":"2024-04-25T10:52:32.473896+0000","last_active":"2024-04-25T10:52:33.791849+0000","last_peered":"2024-04-25T10:52:33.791849+0000","last_clean":"2024-04-25T10:52:33.791849+0000","last_became_active":"2024-04-25T10:52:32.473474+0000","last_became_peered":"2024-04-25T10:52:32.473474+0000","last_unstale":"2024-04-25T10:52:33.791849+0000","last_undegraded":"2024-04-25T10:52:33.791849+0000","last_fullsized":"2024-04-25T10:52:33.791849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:41.568301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348566+0000","last_change":"2024-04-25T10:52:32.476104+0000","last_active":"2024-04-25T10:52:34.348566+0000","last_peered":"2024-04-25T10:52:34.348566+0000","last_clean":"2024-04-25T10:52:34.348566+0000","last_became_active":"2024-04-25T10:52:32.475342+0000","last_became_peered":"2024-04-25T10:52:32.475342+0000","last_unstale":"2024-04-25T10:52:34.348566+0000","last_undegraded":"2024-04-25T10:52:34.348566+0000","last_fullsized":"2024-04-25T10:52:34.348566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:37.087793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348397+0000","last_change":"2024-04-25T10:52:32.470263+0000","last_active":"2024-04-25T10:52:34.348397+0000","last_peered":"2024-04-25T10:52:34.348397+0000","last_clean":"2024-04-25T10:52:34.348397+0000","last_became_active":"2024-04-25T10:52:32.469873+0000","last_became_peered":"2024-04-25T10:52:32.469873+0000","last_unstale":"2024-04-25T10:52:34.348397+0000","last_undegraded":"2024-04-25T10:52:34.348397+0000","last_fullsized":"2024-04-25T10:52:34.348397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.349598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153344+0000","last_change":"2024-04-25T10:52:32.471851+0000","last_active":"2024-04-25T10:52:34.153344+0000","last_peered":"2024-04-25T10:52:34.153344+0000","last_clean":"2024-04-25T10:52:34.153344+0000","last_became_active":"2024-04-25T10:52:32.471477+0000","last_became_peered":"2024-04-25T10:52:32.471477+0000","last_unstale":"2024-04-25T10:52:34.153344+0000","last_undegraded":"2024-04-25T10:52:34.153344+0000","last_fullsized":"2024-04-25T10:52:34.153344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:59.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154824+0000","last_change":"2024-04-25T10:52:32.472530+0000","last_active":"2024-04-25T10:52:34.154824+0000","last_peered":"2024-04-25T10:52:34.154824+0000","last_clean":"2024-04-25T10:52:34.154824+0000","last_became_active":"2024-04-25T10:52:32.472174+0000","last_became_peered":"2024-04-25T10:52:32.472174+0000","last_unstale":"2024-04-25T10:52:34.154824+0000","last_undegraded":"2024-04-25T10:52:34.154824+0000","last_fullsized":"2024-04-25T10:52:34.154824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:09.473575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455379+0000","last_change":"2024-04-25T10:52:32.473617+0000","last_active":"2024-04-25T10:52:33.455379+0000","last_peered":"2024-04-25T10:52:33.455379+0000","last_clean":"2024-04-25T10:52:33.455379+0000","last_became_active":"2024-04-25T10:52:32.473261+0000","last_became_peered":"2024-04-25T10:52:32.473261+0000","last_unstale":"2024-04-25T10:52:33.455379+0000","last_undegraded":"2024-04-25T10:52:33.455379+0000","last_fullsized":"2024-04-25T10:52:33.455379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:05.568672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155275+0000","last_change":"2024-04-25T10:52:32.466296+0000","last_active":"2024-04-25T10:52:34.155275+0000","last_peered":"2024-04-25T10:52:34.155275+0000","last_clean":"2024-04-25T10:52:34.155275+0000","last_became_active":"2024-04-25T10:52:32.463153+0000","last_became_peered":"2024-04-25T10:52:32.463153+0000","last_unstale":"2024-04-25T10:52:34.155275+0000","last_undegraded":"2024-04-25T10:52:34.155275+0000","last_fullsized":"2024-04-25T10:52:34.155275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455290+0000","last_change":"2024-04-25T10:52:31.701949+0000","last_active":"2024-04-25T10:52:33.455290+0000","last_peered":"2024-04-25T10:52:33.455290+0000","last_clean":"2024-04-25T10:52:33.455290+0000","last_became_active":"2024-04-25T10:52:31.701508+0000","last_became_peered":"2024-04-25T10:52:31.701508+0000","last_unstale":"2024-04-25T10:52:33.455290+0000","last_undegraded":"2024-04-25T10:52:33.455290+0000","last_fullsized":"2024-04-25T10:52:33.455290+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:41.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715922+0000","last_change":"2024-04-25T10:52:32.468871+0000","last_active":"2024-04-25T10:52:33.715922+0000","last_peered":"2024-04-25T10:52:33.715922+0000","last_clean":"2024-04-25T10:52:33.715922+0000","last_became_active":"2024-04-25T10:52:32.468460+0000","last_became_peered":"2024-04-25T10:52:32.468460+0000","last_unstale":"2024-04-25T10:52:33.715922+0000","last_undegraded":"2024-04-25T10:52:33.715922+0000","last_fullsized":"2024-04-25T10:52:33.715922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:29.055639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.716131+0000","last_change":"2024-04-25T10:52:31.705068+0000","last_active":"2024-04-25T10:52:33.716131+0000","last_peered":"2024-04-25T10:52:33.716131+0000","last_clean":"2024-04-25T10:52:33.716131+0000","last_became_active":"2024-04-25T10:52:31.704439+0000","last_became_peered":"2024-04-25T10:52:31.704439+0000","last_unstale":"2024-04-25T10:52:33.716131+0000","last_undegraded":"2024-04-25T10:52:33.716131+0000","last_fullsized":"2024-04-25T10:52:33.716131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.347653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348851+0000","last_change":"2024-04-25T10:52:32.476025+0000","last_active":"2024-04-25T10:52:34.348851+0000","last_peered":"2024-04-25T10:52:34.348851+0000","last_clean":"2024-04-25T10:52:34.348851+0000","last_became_active":"2024-04-25T10:52:32.475611+0000","last_became_peered":"2024-04-25T10:52:32.475611+0000","last_unstale":"2024-04-25T10:52:34.348851+0000","last_undegraded":"2024-04-25T10:52:34.348851+0000","last_fullsized":"2024-04-25T10:52:34.348851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:45.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456918+0000","last_change":"2024-04-25T10:52:31.703012+0000","last_active":"2024-04-25T10:52:33.456918+0000","last_peered":"2024-04-25T10:52:33.456918+0000","last_clean":"2024-04-25T10:52:33.456918+0000","last_became_active":"2024-04-25T10:52:31.702689+0000","last_became_peered":"2024-04-25T10:52:31.702689+0000","last_unstale":"2024-04-25T10:52:33.456918+0000","last_undegraded":"2024-04-25T10:52:33.456918+0000","last_fullsized":"2024-04-25T10:52:33.456918+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:13.038859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153493+0000","last_change":"2024-04-25T10:52:32.475858+0000","last_active":"2024-04-25T10:52:34.153493+0000","last_peered":"2024-04-25T10:52:34.153493+0000","last_clean":"2024-04-25T10:52:34.153493+0000","last_became_active":"2024-04-25T10:52:32.475486+0000","last_became_peered":"2024-04-25T10:52:32.475486+0000","last_unstale":"2024-04-25T10:52:34.153493+0000","last_undegraded":"2024-04-25T10:52:34.153493+0000","last_fullsized":"2024-04-25T10:52:34.153493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.439208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456913+0000","last_change":"2024-04-25T10:52:31.698229+0000","last_active":"2024-04-25T10:52:33.456913+0000","last_peered":"2024-04-25T10:52:33.456913+0000","last_clean":"2024-04-25T10:52:33.456913+0000","last_became_active":"2024-04-25T10:52:31.697771+0000","last_became_peered":"2024-04-25T10:52:31.697771+0000","last_unstale":"2024-04-25T10:52:33.456913+0000","last_undegraded":"2024-04-25T10:52:33.456913+0000","last_fullsized":"2024-04-25T10:52:33.456913+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:13.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154668+0000","last_change":"2024-04-25T10:52:32.466495+0000","last_active":"2024-04-25T10:52:34.154668+0000","last_peered":"2024-04-25T10:52:34.154668+0000","last_clean":"2024-04-25T10:52:34.154668+0000","last_became_active":"2024-04-25T10:52:32.462400+0000","last_became_peered":"2024-04-25T10:52:32.462400+0000","last_unstale":"2024-04-25T10:52:34.154668+0000","last_undegraded":"2024-04-25T10:52:34.154668+0000","last_fullsized":"2024-04-25T10:52:34.154668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:18.770981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154306+0000","last_change":"2024-04-25T10:52:31.699644+0000","last_active":"2024-04-25T10:52:34.154306+0000","last_peered":"2024-04-25T10:52:34.154306+0000","last_clean":"2024-04-25T10:52:34.154306+0000","last_became_active":"2024-04-25T10:52:31.699315+0000","last_became_peered":"2024-04-25T10:52:31.699315+0000","last_unstale":"2024-04-25T10:52:34.154306+0000","last_undegraded":"2024-04-25T10:52:34.154306+0000","last_fullsized":"2024-04-25T10:52:34.154306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:52.915096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456333+0000","last_change":"2024-04-25T10:52:32.463372+0000","last_active":"2024-04-25T10:52:33.456333+0000","last_peered":"2024-04-25T10:52:33.456333+0000","last_clean":"2024-04-25T10:52:33.456333+0000","last_became_active":"2024-04-25T10:52:32.462926+0000","last_became_peered":"2024-04-25T10:52:32.462926+0000","last_unstale":"2024-04-25T10:52:33.456333+0000","last_undegraded":"2024-04-25T10:52:33.456333+0000","last_fullsized":"2024-04-25T10:52:33.456333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:17.724268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153916+0000","last_change":"2024-04-25T10:52:31.695754+0000","last_active":"2024-04-25T10:52:34.153916+0000","last_peered":"2024-04-25T10:52:34.153916+0000","last_clean":"2024-04-25T10:52:34.153916+0000","last_became_active":"2024-04-25T10:52:31.695381+0000","last_became_peered":"2024-04-25T10:52:31.695381+0000","last_unstale":"2024-04-25T10:52:34.153916+0000","last_undegraded":"2024-04-25T10:52:34.153916+0000","last_fullsized":"2024-04-25T10:52:34.153916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:58.851534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154584+0000","last_change":"2024-04-25T10:52:32.461959+0000","last_active":"2024-04-25T10:52:34.154584+0000","last_peered":"2024-04-25T10:52:34.154584+0000","last_clean":"2024-04-25T10:52:34.154584+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:52:34.154584+0000","last_undegraded":"2024-04-25T10:52:34.154584+0000","last_fullsized":"2024-04-25T10:52:34.154584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:28.238824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791672+0000","last_change":"2024-04-25T10:52:31.695067+0000","last_active":"2024-04-25T10:52:33.791672+0000","last_peered":"2024-04-25T10:52:33.791672+0000","last_clean":"2024-04-25T10:52:33.791672+0000","last_became_active":"2024-04-25T10:52:31.694648+0000","last_became_peered":"2024-04-25T10:52:31.694648+0000","last_unstale":"2024-04-25T10:52:33.791672+0000","last_undegraded":"2024-04-25T10:52:33.791672+0000","last_fullsized":"2024-04-25T10:52:33.791672+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:09.654835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.714906+0000","last_change":"2024-04-25T10:52:32.453501+0000","last_active":"2024-04-25T10:52:33.714906+0000","last_peered":"2024-04-25T10:52:33.714906+0000","last_clean":"2024-04-25T10:52:33.714906+0000","last_became_active":"2024-04-25T10:52:32.453113+0000","last_became_peered":"2024-04-25T10:52:32.453113+0000","last_unstale":"2024-04-25T10:52:33.714906+0000","last_undegraded":"2024-04-25T10:52:33.714906+0000","last_fullsized":"2024-04-25T10:52:33.714906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:07.340709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153710+0000","last_change":"2024-04-25T10:52:31.697946+0000","last_active":"2024-04-25T10:52:34.153710+0000","last_peered":"2024-04-25T10:52:34.153710+0000","last_clean":"2024-04-25T10:52:34.153710+0000","last_became_active":"2024-04-25T10:52:31.697612+0000","last_became_peered":"2024-04-25T10:52:31.697612+0000","last_unstale":"2024-04-25T10:52:34.153710+0000","last_undegraded":"2024-04-25T10:52:34.153710+0000","last_fullsized":"2024-04-25T10:52:34.153710+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.280776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455450+0000","last_change":"2024-04-25T10:52:32.473537+0000","last_active":"2024-04-25T10:52:33.455450+0000","last_peered":"2024-04-25T10:52:33.455450+0000","last_clean":"2024-04-25T10:52:33.455450+0000","last_became_active":"2024-04-25T10:52:32.473164+0000","last_became_peered":"2024-04-25T10:52:32.473164+0000","last_unstale":"2024-04-25T10:52:33.455450+0000","last_undegraded":"2024-04-25T10:52:33.455450+0000","last_fullsized":"2024-04-25T10:52:33.455450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.008031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154421+0000","last_change":"2024-04-25T10:52:31.694207+0000","last_active":"2024-04-25T10:52:34.154421+0000","last_peered":"2024-04-25T10:52:34.154421+0000","last_clean":"2024-04-25T10:52:34.154421+0000","last_became_active":"2024-04-25T10:52:31.693777+0000","last_became_peered":"2024-04-25T10:52:31.693777+0000","last_unstale":"2024-04-25T10:52:34.154421+0000","last_undegraded":"2024-04-25T10:52:34.154421+0000","last_fullsized":"2024-04-25T10:52:34.154421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:43.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154422+0000","last_change":"2024-04-25T10:52:32.475545+0000","last_active":"2024-04-25T10:52:34.154422+0000","last_peered":"2024-04-25T10:52:34.154422+0000","last_clean":"2024-04-25T10:52:34.154422+0000","last_became_active":"2024-04-25T10:52:32.475195+0000","last_became_peered":"2024-04-25T10:52:32.475195+0000","last_unstale":"2024-04-25T10:52:34.154422+0000","last_undegraded":"2024-04-25T10:52:34.154422+0000","last_fullsized":"2024-04-25T10:52:34.154422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:49.900374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155036+0000","last_change":"2024-04-25T10:52:31.698184+0000","last_active":"2024-04-25T10:52:34.155036+0000","last_peered":"2024-04-25T10:52:34.155036+0000","last_clean":"2024-04-25T10:52:34.155036+0000","last_became_active":"2024-04-25T10:52:31.697816+0000","last_became_peered":"2024-04-25T10:52:31.697816+0000","last_unstale":"2024-04-25T10:52:34.155036+0000","last_undegraded":"2024-04-25T10:52:34.155036+0000","last_fullsized":"2024-04-25T10:52:34.155036+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:44.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348894+0000","last_change":"2024-04-25T10:52:32.472437+0000","last_active":"2024-04-25T10:52:34.348894+0000","last_peered":"2024-04-25T10:52:34.348894+0000","last_clean":"2024-04-25T10:52:34.348894+0000","last_became_active":"2024-04-25T10:52:32.471944+0000","last_became_peered":"2024-04-25T10:52:32.471944+0000","last_unstale":"2024-04-25T10:52:34.348894+0000","last_undegraded":"2024-04-25T10:52:34.348894+0000","last_fullsized":"2024-04-25T10:52:34.348894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:11.047158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154758+0000","last_change":"2024-04-25T10:52:31.696943+0000","last_active":"2024-04-25T10:52:34.154758+0000","last_peered":"2024-04-25T10:52:34.154758+0000","last_clean":"2024-04-25T10:52:34.154758+0000","last_became_active":"2024-04-25T10:52:31.696561+0000","last_became_peered":"2024-04-25T10:52:31.696561+0000","last_unstale":"2024-04-25T10:52:34.154758+0000","last_undegraded":"2024-04-25T10:52:34.154758+0000","last_fullsized":"2024-04-25T10:52:34.154758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:14.628800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153856+0000","last_change":"2024-04-25T10:52:32.461966+0000","last_active":"2024-04-25T10:52:34.153856+0000","last_peered":"2024-04-25T10:52:34.153856+0000","last_clean":"2024-04-25T10:52:34.153856+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:52:34.153856+0000","last_undegraded":"2024-04-25T10:52:34.153856+0000","last_fullsized":"2024-04-25T10:52:34.153856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'6","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154245+0000","last_change":"2024-04-25T10:52:31.690938+0000","last_active":"2024-04-25T10:52:34.154245+0000","last_peered":"2024-04-25T10:52:34.154245+0000","last_clean":"2024-04-25T10:52:34.154245+0000","last_became_active":"2024-04-25T10:52:31.690511+0000","last_became_peered":"2024-04-25T10:52:31.690511+0000","last_unstale":"2024-04-25T10:52:34.154245+0000","last_undegraded":"2024-04-25T10:52:34.154245+0000","last_fullsized":"2024-04-25T10:52:34.154245+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:10.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456758+0000","last_change":"2024-04-25T10:52:32.474871+0000","last_active":"2024-04-25T10:52:33.456758+0000","last_peered":"2024-04-25T10:52:33.456758+0000","last_clean":"2024-04-25T10:52:33.456758+0000","last_became_active":"2024-04-25T10:52:32.474463+0000","last_became_peered":"2024-04-25T10:52:32.474463+0000","last_unstale":"2024-04-25T10:52:33.456758+0000","last_undegraded":"2024-04-25T10:52:33.456758+0000","last_fullsized":"2024-04-25T10:52:33.456758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:36.719230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'2","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154653+0000","last_change":"2024-04-25T10:52:31.691006+0000","last_active":"2024-04-25T10:52:34.154653+0000","last_peered":"2024-04-25T10:52:34.154653+0000","last_clean":"2024-04-25T10:52:34.154653+0000","last_became_active":"2024-04-25T10:52:31.690611+0000","last_became_peered":"2024-04-25T10:52:31.690611+0000","last_unstale":"2024-04-25T10:52:34.154653+0000","last_undegraded":"2024-04-25T10:52:34.154653+0000","last_fullsized":"2024-04-25T10:52:34.154653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:37.781160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791737+0000","last_change":"2024-04-25T10:52:32.463331+0000","last_active":"2024-04-25T10:52:33.791737+0000","last_peered":"2024-04-25T10:52:33.791737+0000","last_clean":"2024-04-25T10:52:33.791737+0000","last_became_active":"2024-04-25T10:52:32.462579+0000","last_became_peered":"2024-04-25T10:52:32.462579+0000","last_unstale":"2024-04-25T10:52:33.791737+0000","last_undegraded":"2024-04-25T10:52:33.791737+0000","last_fullsized":"2024-04-25T10:52:33.791737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154357+0000","last_change":"2024-04-25T10:52:31.704675+0000","last_active":"2024-04-25T10:52:34.154357+0000","last_peered":"2024-04-25T10:52:34.154357+0000","last_clean":"2024-04-25T10:52:34.154357+0000","last_became_active":"2024-04-25T10:52:31.704236+0000","last_became_peered":"2024-04-25T10:52:31.704236+0000","last_unstale":"2024-04-25T10:52:34.154357+0000","last_undegraded":"2024-04-25T10:52:34.154357+0000","last_fullsized":"2024-04-25T10:52:34.154357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.637006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155385+0000","last_change":"2024-04-25T10:52:32.462960+0000","last_active":"2024-04-25T10:52:34.155385+0000","last_peered":"2024-04-25T10:52:34.155385+0000","last_clean":"2024-04-25T10:52:34.155385+0000","last_became_active":"2024-04-25T10:52:32.461993+0000","last_became_peered":"2024-04-25T10:52:32.461993+0000","last_unstale":"2024-04-25T10:52:34.155385+0000","last_undegraded":"2024-04-25T10:52:34.155385+0000","last_fullsized":"2024-04-25T10:52:34.155385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:01.803209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456118+0000","last_change":"2024-04-25T10:52:31.698206+0000","last_active":"2024-04-25T10:52:33.456118+0000","last_peered":"2024-04-25T10:52:33.456118+0000","last_clean":"2024-04-25T10:52:33.456118+0000","last_became_active":"2024-04-25T10:52:31.697814+0000","last_became_peered":"2024-04-25T10:52:31.697814+0000","last_unstale":"2024-04-25T10:52:33.456118+0000","last_undegraded":"2024-04-25T10:52:33.456118+0000","last_fullsized":"2024-04-25T10:52:33.456118+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:02.849229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154769+0000","last_change":"2024-04-25T10:52:32.473551+0000","last_active":"2024-04-25T10:52:34.154769+0000","last_peered":"2024-04-25T10:52:34.154769+0000","last_clean":"2024-04-25T10:52:34.154769+0000","last_became_active":"2024-04-25T10:52:32.473227+0000","last_became_peered":"2024-04-25T10:52:32.473227+0000","last_unstale":"2024-04-25T10:52:34.154769+0000","last_undegraded":"2024-04-25T10:52:34.154769+0000","last_fullsized":"2024-04-25T10:52:34.154769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:37.742066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'8","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456375+0000","last_change":"2024-04-25T10:52:31.700670+0000","last_active":"2024-04-25T10:52:33.456375+0000","last_peered":"2024-04-25T10:52:33.456375+0000","last_clean":"2024-04-25T10:52:33.456375+0000","last_became_active":"2024-04-25T10:52:31.700234+0000","last_became_peered":"2024-04-25T10:52:31.700234+0000","last_unstale":"2024-04-25T10:52:33.456375+0000","last_undegraded":"2024-04-25T10:52:33.456375+0000","last_fullsized":"2024-04-25T10:52:33.456375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:21.795784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154852+0000","last_change":"2024-04-25T10:52:32.463452+0000","last_active":"2024-04-25T10:52:34.154852+0000","last_peered":"2024-04-25T10:52:34.154852+0000","last_clean":"2024-04-25T10:52:34.154852+0000","last_became_active":"2024-04-25T10:52:32.461668+0000","last_became_peered":"2024-04-25T10:52:32.461668+0000","last_unstale":"2024-04-25T10:52:34.154852+0000","last_undegraded":"2024-04-25T10:52:34.154852+0000","last_fullsized":"2024-04-25T10:52:34.154852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.871762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456702+0000","last_change":"2024-04-25T10:52:31.698507+0000","last_active":"2024-04-25T10:52:33.456702+0000","last_peered":"2024-04-25T10:52:33.456702+0000","last_clean":"2024-04-25T10:52:33.456702+0000","last_became_active":"2024-04-25T10:52:31.698148+0000","last_became_peered":"2024-04-25T10:52:31.698148+0000","last_unstale":"2024-04-25T10:52:33.456702+0000","last_undegraded":"2024-04-25T10:52:33.456702+0000","last_fullsized":"2024-04-25T10:52:33.456702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:48.993234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154021+0000","last_change":"2024-04-25T10:52:32.465874+0000","last_active":"2024-04-25T10:52:34.154021+0000","last_peered":"2024-04-25T10:52:34.154021+0000","last_clean":"2024-04-25T10:52:34.154021+0000","last_became_active":"2024-04-25T10:52:32.465511+0000","last_became_peered":"2024-04-25T10:52:32.465511+0000","last_unstale":"2024-04-25T10:52:34.154021+0000","last_undegraded":"2024-04-25T10:52:34.154021+0000","last_fullsized":"2024-04-25T10:52:34.154021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:15.274909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.455263+0000","last_change":"2024-04-25T10:52:31.691074+0000","last_active":"2024-04-25T10:52:33.455263+0000","last_peered":"2024-04-25T10:52:33.455263+0000","last_clean":"2024-04-25T10:52:33.455263+0000","last_became_active":"2024-04-25T10:52:31.690402+0000","last_became_peered":"2024-04-25T10:52:31.690402+0000","last_unstale":"2024-04-25T10:52:33.455263+0000","last_undegraded":"2024-04-25T10:52:33.455263+0000","last_fullsized":"2024-04-25T10:52:33.455263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.529860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154509+0000","last_change":"2024-04-25T10:52:32.477968+0000","last_active":"2024-04-25T10:52:34.154509+0000","last_peered":"2024-04-25T10:52:34.154509+0000","last_clean":"2024-04-25T10:52:34.154509+0000","last_became_active":"2024-04-25T10:52:32.476395+0000","last_became_peered":"2024-04-25T10:52:32.476395+0000","last_unstale":"2024-04-25T10:52:34.154509+0000","last_undegraded":"2024-04-25T10:52:34.154509+0000","last_fullsized":"2024-04-25T10:52:34.154509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:13.128910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154891+0000","last_change":"2024-04-25T10:52:31.698934+0000","last_active":"2024-04-25T10:52:34.154891+0000","last_peered":"2024-04-25T10:52:34.154891+0000","last_clean":"2024-04-25T10:52:34.154891+0000","last_became_active":"2024-04-25T10:52:31.698565+0000","last_became_peered":"2024-04-25T10:52:31.698565+0000","last_unstale":"2024-04-25T10:52:34.154891+0000","last_undegraded":"2024-04-25T10:52:34.154891+0000","last_fullsized":"2024-04-25T10:52:34.154891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.360617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154004+0000","last_change":"2024-04-25T10:52:32.477538+0000","last_active":"2024-04-25T10:52:34.154004+0000","last_peered":"2024-04-25T10:52:34.154004+0000","last_clean":"2024-04-25T10:52:34.154004+0000","last_became_active":"2024-04-25T10:52:32.476088+0000","last_became_peered":"2024-04-25T10:52:32.476088+0000","last_unstale":"2024-04-25T10:52:34.154004+0000","last_undegraded":"2024-04-25T10:52:34.154004+0000","last_fullsized":"2024-04-25T10:52:34.154004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.987071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154129+0000","last_change":"2024-04-25T10:52:32.469335+0000","last_active":"2024-04-25T10:52:34.154129+0000","last_peered":"2024-04-25T10:52:34.154129+0000","last_clean":"2024-04-25T10:52:34.154129+0000","last_became_active":"2024-04-25T10:52:32.468921+0000","last_became_peered":"2024-04-25T10:52:32.468921+0000","last_unstale":"2024-04-25T10:52:34.154129+0000","last_undegraded":"2024-04-25T10:52:34.154129+0000","last_fullsized":"2024-04-25T10:52:34.154129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:26.499243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791540+0000","last_change":"2024-04-25T10:52:31.706700+0000","last_active":"2024-04-25T10:52:33.791540+0000","last_peered":"2024-04-25T10:52:33.791540+0000","last_clean":"2024-04-25T10:52:33.791540+0000","last_became_active":"2024-04-25T10:52:31.706392+0000","last_became_peered":"2024-04-25T10:52:31.706392+0000","last_unstale":"2024-04-25T10:52:33.791540+0000","last_undegraded":"2024-04-25T10:52:33.791540+0000","last_fullsized":"2024-04-25T10:52:33.791540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:58.867351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715326+0000","last_change":"2024-04-25T10:52:32.462603+0000","last_active":"2024-04-25T10:52:33.715326+0000","last_peered":"2024-04-25T10:52:33.715326+0000","last_clean":"2024-04-25T10:52:33.715326+0000","last_became_active":"2024-04-25T10:52:32.462238+0000","last_became_peered":"2024-04-25T10:52:32.462238+0000","last_unstale":"2024-04-25T10:52:33.715326+0000","last_undegraded":"2024-04-25T10:52:33.715326+0000","last_fullsized":"2024-04-25T10:52:33.715326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.555683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456322+0000","last_change":"2024-04-25T10:52:31.694054+0000","last_active":"2024-04-25T10:52:33.456322+0000","last_peered":"2024-04-25T10:52:33.456322+0000","last_clean":"2024-04-25T10:52:33.456322+0000","last_became_active":"2024-04-25T10:52:31.693693+0000","last_became_peered":"2024-04-25T10:52:31.693693+0000","last_unstale":"2024-04-25T10:52:33.456322+0000","last_undegraded":"2024-04-25T10:52:33.456322+0000","last_fullsized":"2024-04-25T10:52:33.456322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:22.238131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154745+0000","last_change":"2024-04-25T10:51:22.730925+0000","last_active":"2024-04-25T10:52:34.154745+0000","last_peered":"2024-04-25T10:52:34.154745+0000","last_clean":"2024-04-25T10:52:34.154745+0000","last_became_active":"2024-04-25T10:51:22.725168+0000","last_became_peered":"2024-04-25T10:51:22.725168+0000","last_unstale":"2024-04-25T10:52:34.154745+0000","last_undegraded":"2024-04-25T10:52:34.154745+0000","last_fullsized":"2024-04-25T10:52:34.154745+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_clean_scrub_stamp":"2024-04-25T10:47:56.758246+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:02.416333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791576+0000","last_change":"2024-04-25T10:52:32.461608+0000","last_active":"2024-04-25T10:52:33.791576+0000","last_peered":"2024-04-25T10:52:33.791576+0000","last_clean":"2024-04-25T10:52:33.791576+0000","last_became_active":"2024-04-25T10:52:32.461210+0000","last_became_peered":"2024-04-25T10:52:32.461210+0000","last_unstale":"2024-04-25T10:52:33.791576+0000","last_undegraded":"2024-04-25T10:52:33.791576+0000","last_fullsized":"2024-04-25T10:52:33.791576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:14.897053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154935+0000","last_change":"2024-04-25T10:52:31.701003+0000","last_active":"2024-04-25T10:52:34.154935+0000","last_peered":"2024-04-25T10:52:34.154935+0000","last_clean":"2024-04-25T10:52:34.154935+0000","last_became_active":"2024-04-25T10:52:31.699623+0000","last_became_peered":"2024-04-25T10:52:31.699623+0000","last_unstale":"2024-04-25T10:52:34.154935+0000","last_undegraded":"2024-04-25T10:52:34.154935+0000","last_fullsized":"2024-04-25T10:52:34.154935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:45.175760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456723+0000","last_change":"2024-04-25T10:52:31.702595+0000","last_active":"2024-04-25T10:52:33.456723+0000","last_peered":"2024-04-25T10:52:33.456723+0000","last_clean":"2024-04-25T10:52:33.456723+0000","last_became_active":"2024-04-25T10:52:31.702124+0000","last_became_peered":"2024-04-25T10:52:31.702124+0000","last_unstale":"2024-04-25T10:52:33.456723+0000","last_undegraded":"2024-04-25T10:52:33.456723+0000","last_fullsized":"2024-04-25T10:52:33.456723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:54.351859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.792032+0000","last_change":"2024-04-25T10:52:32.467987+0000","last_active":"2024-04-25T10:52:33.792032+0000","last_peered":"2024-04-25T10:52:33.792032+0000","last_clean":"2024-04-25T10:52:33.792032+0000","last_became_active":"2024-04-25T10:52:32.467613+0000","last_became_peered":"2024-04-25T10:52:32.467613+0000","last_unstale":"2024-04-25T10:52:33.792032+0000","last_undegraded":"2024-04-25T10:52:33.792032+0000","last_fullsized":"2024-04-25T10:52:33.792032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:00.112102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.713033+0000","last_change":"2024-04-25T10:52:31.699868+0000","last_active":"2024-04-25T10:52:33.713033+0000","last_peered":"2024-04-25T10:52:33.713033+0000","last_clean":"2024-04-25T10:52:33.713033+0000","last_became_active":"2024-04-25T10:52:31.699470+0000","last_became_peered":"2024-04-25T10:52:31.699470+0000","last_unstale":"2024-04-25T10:52:33.713033+0000","last_undegraded":"2024-04-25T10:52:33.713033+0000","last_fullsized":"2024-04-25T10:52:33.713033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.161821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715642+0000","last_change":"2024-04-25T10:52:32.459082+0000","last_active":"2024-04-25T10:52:33.715642+0000","last_peered":"2024-04-25T10:52:33.715642+0000","last_clean":"2024-04-25T10:52:33.715642+0000","last_became_active":"2024-04-25T10:52:32.458190+0000","last_became_peered":"2024-04-25T10:52:32.458190+0000","last_unstale":"2024-04-25T10:52:33.715642+0000","last_undegraded":"2024-04-25T10:52:33.715642+0000","last_fullsized":"2024-04-25T10:52:33.715642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.280762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456674+0000","last_change":"2024-04-25T10:52:31.700636+0000","last_active":"2024-04-25T10:52:33.456674+0000","last_peered":"2024-04-25T10:52:33.456674+0000","last_clean":"2024-04-25T10:52:33.456674+0000","last_became_active":"2024-04-25T10:52:31.700268+0000","last_became_peered":"2024-04-25T10:52:31.700268+0000","last_unstale":"2024-04-25T10:52:33.456674+0000","last_undegraded":"2024-04-25T10:52:33.456674+0000","last_fullsized":"2024-04-25T10:52:33.456674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.439491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348914+0000","last_change":"2024-04-25T10:52:32.467406+0000","last_active":"2024-04-25T10:52:34.348914+0000","last_peered":"2024-04-25T10:52:34.348914+0000","last_clean":"2024-04-25T10:52:34.348914+0000","last_became_active":"2024-04-25T10:52:32.467053+0000","last_became_peered":"2024-04-25T10:52:32.467053+0000","last_unstale":"2024-04-25T10:52:34.348914+0000","last_undegraded":"2024-04-25T10:52:34.348914+0000","last_fullsized":"2024-04-25T10:52:34.348914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153258+0000","last_change":"2024-04-25T10:52:31.689623+0000","last_active":"2024-04-25T10:52:34.153258+0000","last_peered":"2024-04-25T10:52:34.153258+0000","last_clean":"2024-04-25T10:52:34.153258+0000","last_became_active":"2024-04-25T10:52:31.689261+0000","last_became_peered":"2024-04-25T10:52:31.689261+0000","last_unstale":"2024-04-25T10:52:34.153258+0000","last_undegraded":"2024-04-25T10:52:34.153258+0000","last_fullsized":"2024-04-25T10:52:34.153258+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:42.787964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456293+0000","last_change":"2024-04-25T10:52:32.472709+0000","last_active":"2024-04-25T10:52:33.456293+0000","last_peered":"2024-04-25T10:52:33.456293+0000","last_clean":"2024-04-25T10:52:33.456293+0000","last_became_active":"2024-04-25T10:52:32.472331+0000","last_became_peered":"2024-04-25T10:52:32.472331+0000","last_unstale":"2024-04-25T10:52:33.456293+0000","last_undegraded":"2024-04-25T10:52:33.456293+0000","last_fullsized":"2024-04-25T10:52:33.456293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.047682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715149+0000","last_change":"2024-04-25T10:52:31.697360+0000","last_active":"2024-04-25T10:52:33.715149+0000","last_peered":"2024-04-25T10:52:33.715149+0000","last_clean":"2024-04-25T10:52:33.715149+0000","last_became_active":"2024-04-25T10:52:31.696991+0000","last_became_peered":"2024-04-25T10:52:31.696991+0000","last_unstale":"2024-04-25T10:52:33.715149+0000","last_undegraded":"2024-04-25T10:52:33.715149+0000","last_fullsized":"2024-04-25T10:52:33.715149+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:38.395372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153990+0000","last_change":"2024-04-25T10:52:32.472505+0000","last_active":"2024-04-25T10:52:34.153990+0000","last_peered":"2024-04-25T10:52:34.153990+0000","last_clean":"2024-04-25T10:52:34.153990+0000","last_became_active":"2024-04-25T10:52:32.472113+0000","last_became_peered":"2024-04-25T10:52:32.472113+0000","last_unstale":"2024-04-25T10:52:34.153990+0000","last_undegraded":"2024-04-25T10:52:34.153990+0000","last_fullsized":"2024-04-25T10:52:34.153990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:20.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154274+0000","last_change":"2024-04-25T10:52:31.696488+0000","last_active":"2024-04-25T10:52:34.154274+0000","last_peered":"2024-04-25T10:52:34.154274+0000","last_clean":"2024-04-25T10:52:34.154274+0000","last_became_active":"2024-04-25T10:52:31.696062+0000","last_became_peered":"2024-04-25T10:52:31.696062+0000","last_unstale":"2024-04-25T10:52:34.154274+0000","last_undegraded":"2024-04-25T10:52:34.154274+0000","last_fullsized":"2024-04-25T10:52:34.154274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:24.329938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715801+0000","last_change":"2024-04-25T10:52:32.474555+0000","last_active":"2024-04-25T10:52:33.715801+0000","last_peered":"2024-04-25T10:52:33.715801+0000","last_clean":"2024-04-25T10:52:33.715801+0000","last_became_active":"2024-04-25T10:52:32.474001+0000","last_became_peered":"2024-04-25T10:52:32.474001+0000","last_unstale":"2024-04-25T10:52:33.715801+0000","last_undegraded":"2024-04-25T10:52:33.715801+0000","last_fullsized":"2024-04-25T10:52:33.715801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.775966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'2","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.456431+0000","last_change":"2024-04-25T10:52:31.702251+0000","last_active":"2024-04-25T10:52:33.456431+0000","last_peered":"2024-04-25T10:52:33.456431+0000","last_clean":"2024-04-25T10:52:33.456431+0000","last_became_active":"2024-04-25T10:52:31.701763+0000","last_became_peered":"2024-04-25T10:52:31.701763+0000","last_unstale":"2024-04-25T10:52:33.456431+0000","last_undegraded":"2024-04-25T10:52:33.456431+0000","last_fullsized":"2024-04-25T10:52:33.456431+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155330+0000","last_change":"2024-04-25T10:52:32.465494+0000","last_active":"2024-04-25T10:52:34.155330+0000","last_peered":"2024-04-25T10:52:34.155330+0000","last_clean":"2024-04-25T10:52:34.155330+0000","last_became_active":"2024-04-25T10:52:32.463855+0000","last_became_peered":"2024-04-25T10:52:32.463855+0000","last_unstale":"2024-04-25T10:52:34.155330+0000","last_undegraded":"2024-04-25T10:52:34.155330+0000","last_fullsized":"2024-04-25T10:52:34.155330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:40.970580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155104+0000","last_change":"2024-04-25T10:52:31.700749+0000","last_active":"2024-04-25T10:52:34.155104+0000","last_peered":"2024-04-25T10:52:34.155104+0000","last_clean":"2024-04-25T10:52:34.155104+0000","last_became_active":"2024-04-25T10:52:31.699109+0000","last_became_peered":"2024-04-25T10:52:31.699109+0000","last_unstale":"2024-04-25T10:52:34.155104+0000","last_undegraded":"2024-04-25T10:52:34.155104+0000","last_fullsized":"2024-04-25T10:52:34.155104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:48.405333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155314+0000","last_change":"2024-04-25T10:52:32.468292+0000","last_active":"2024-04-25T10:52:34.155314+0000","last_peered":"2024-04-25T10:52:34.155314+0000","last_clean":"2024-04-25T10:52:34.155314+0000","last_became_active":"2024-04-25T10:52:32.467934+0000","last_became_peered":"2024-04-25T10:52:32.467934+0000","last_unstale":"2024-04-25T10:52:34.155314+0000","last_undegraded":"2024-04-25T10:52:34.155314+0000","last_fullsized":"2024-04-25T10:52:34.155314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.366735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154543+0000","last_change":"2024-04-25T10:52:31.705025+0000","last_active":"2024-04-25T10:52:34.154543+0000","last_peered":"2024-04-25T10:52:34.154543+0000","last_clean":"2024-04-25T10:52:34.154543+0000","last_became_active":"2024-04-25T10:52:31.704532+0000","last_became_peered":"2024-04-25T10:52:31.704532+0000","last_unstale":"2024-04-25T10:52:34.154543+0000","last_undegraded":"2024-04-25T10:52:34.154543+0000","last_fullsized":"2024-04-25T10:52:34.154543+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.371778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155155+0000","last_change":"2024-04-25T10:52:32.464149+0000","last_active":"2024-04-25T10:52:34.155155+0000","last_peered":"2024-04-25T10:52:34.155155+0000","last_clean":"2024-04-25T10:52:34.155155+0000","last_became_active":"2024-04-25T10:52:32.463630+0000","last_became_peered":"2024-04-25T10:52:32.463630+0000","last_unstale":"2024-04-25T10:52:34.155155+0000","last_undegraded":"2024-04-25T10:52:34.155155+0000","last_fullsized":"2024-04-25T10:52:34.155155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:38.219914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791752+0000","last_change":"2024-04-25T10:52:31.705770+0000","last_active":"2024-04-25T10:52:33.791752+0000","last_peered":"2024-04-25T10:52:33.791752+0000","last_clean":"2024-04-25T10:52:33.791752+0000","last_became_active":"2024-04-25T10:52:31.705391+0000","last_became_peered":"2024-04-25T10:52:31.705391+0000","last_unstale":"2024-04-25T10:52:33.791752+0000","last_undegraded":"2024-04-25T10:52:33.791752+0000","last_fullsized":"2024-04-25T10:52:33.791752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:32.831435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155504+0000","last_change":"2024-04-25T10:52:32.465550+0000","last_active":"2024-04-25T10:52:34.155504+0000","last_peered":"2024-04-25T10:52:34.155504+0000","last_clean":"2024-04-25T10:52:34.155504+0000","last_became_active":"2024-04-25T10:52:32.465034+0000","last_became_peered":"2024-04-25T10:52:32.465034+0000","last_unstale":"2024-04-25T10:52:34.155504+0000","last_undegraded":"2024-04-25T10:52:34.155504+0000","last_fullsized":"2024-04-25T10:52:34.155504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.810469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154283+0000","last_change":"2024-04-25T10:52:32.470812+0000","last_active":"2024-04-25T10:52:34.154283+0000","last_peered":"2024-04-25T10:52:34.154283+0000","last_clean":"2024-04-25T10:52:34.154283+0000","last_became_active":"2024-04-25T10:52:32.469620+0000","last_became_peered":"2024-04-25T10:52:32.469620+0000","last_unstale":"2024-04-25T10:52:34.154283+0000","last_undegraded":"2024-04-25T10:52:34.154283+0000","last_fullsized":"2024-04-25T10:52:34.154283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154471+0000","last_change":"2024-04-25T10:52:32.464762+0000","last_active":"2024-04-25T10:52:34.154471+0000","last_peered":"2024-04-25T10:52:34.154471+0000","last_clean":"2024-04-25T10:52:34.154471+0000","last_became_active":"2024-04-25T10:52:32.464316+0000","last_became_peered":"2024-04-25T10:52:32.464316+0000","last_unstale":"2024-04-25T10:52:34.154471+0000","last_undegraded":"2024-04-25T10:52:34.154471+0000","last_fullsized":"2024-04-25T10:52:34.154471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:55.526572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154197+0000","last_change":"2024-04-25T10:52:32.466429+0000","last_active":"2024-04-25T10:52:34.154197+0000","last_peered":"2024-04-25T10:52:34.154197+0000","last_clean":"2024-04-25T10:52:34.154197+0000","last_became_active":"2024-04-25T10:52:32.465132+0000","last_became_peered":"2024-04-25T10:52:32.465132+0000","last_unstale":"2024-04-25T10:52:34.154197+0000","last_undegraded":"2024-04-25T10:52:34.154197+0000","last_fullsized":"2024-04-25T10:52:34.154197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:14.274441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.348595+0000","last_change":"2024-04-25T10:52:32.464666+0000","last_active":"2024-04-25T10:52:34.348595+0000","last_peered":"2024-04-25T10:52:34.348595+0000","last_clean":"2024-04-25T10:52:34.348595+0000","last_became_active":"2024-04-25T10:52:32.464226+0000","last_became_peered":"2024-04-25T10:52:32.464226+0000","last_unstale":"2024-04-25T10:52:34.348595+0000","last_undegraded":"2024-04-25T10:52:34.348595+0000","last_fullsized":"2024-04-25T10:52:34.348595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:39.388542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791932+0000","last_change":"2024-04-25T10:52:32.465199+0000","last_active":"2024-04-25T10:52:33.791932+0000","last_peered":"2024-04-25T10:52:33.791932+0000","last_clean":"2024-04-25T10:52:33.791932+0000","last_became_active":"2024-04-25T10:52:32.464285+0000","last_became_peered":"2024-04-25T10:52:32.464285+0000","last_unstale":"2024-04-25T10:52:33.791932+0000","last_undegraded":"2024-04-25T10:52:33.791932+0000","last_fullsized":"2024-04-25T10:52:33.791932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.010994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154224+0000","last_change":"2024-04-25T10:52:32.469140+0000","last_active":"2024-04-25T10:52:34.154224+0000","last_peered":"2024-04-25T10:52:34.154224+0000","last_clean":"2024-04-25T10:52:34.154224+0000","last_became_active":"2024-04-25T10:52:32.468344+0000","last_became_peered":"2024-04-25T10:52:32.468344+0000","last_unstale":"2024-04-25T10:52:34.154224+0000","last_undegraded":"2024-04-25T10:52:34.154224+0000","last_fullsized":"2024-04-25T10:52:34.154224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:07.228059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.155386+0000","last_change":"2024-04-25T10:52:32.469326+0000","last_active":"2024-04-25T10:52:34.155386+0000","last_peered":"2024-04-25T10:52:34.155386+0000","last_clean":"2024-04-25T10:52:34.155386+0000","last_became_active":"2024-04-25T10:52:32.468905+0000","last_became_peered":"2024-04-25T10:52:32.468905+0000","last_unstale":"2024-04-25T10:52:34.155386+0000","last_undegraded":"2024-04-25T10:52:34.155386+0000","last_fullsized":"2024-04-25T10:52:34.155386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:31.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715511+0000","last_change":"2024-04-25T10:52:32.459391+0000","last_active":"2024-04-25T10:52:33.715511+0000","last_peered":"2024-04-25T10:52:33.715511+0000","last_clean":"2024-04-25T10:52:33.715511+0000","last_became_active":"2024-04-25T10:52:32.458702+0000","last_became_peered":"2024-04-25T10:52:32.458702+0000","last_unstale":"2024-04-25T10:52:33.715511+0000","last_undegraded":"2024-04-25T10:52:33.715511+0000","last_fullsized":"2024-04-25T10:52:33.715511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.475714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153957+0000","last_change":"2024-04-25T10:52:32.467395+0000","last_active":"2024-04-25T10:52:34.153957+0000","last_peered":"2024-04-25T10:52:34.153957+0000","last_clean":"2024-04-25T10:52:34.153957+0000","last_became_active":"2024-04-25T10:52:32.466976+0000","last_became_peered":"2024-04-25T10:52:32.466976+0000","last_unstale":"2024-04-25T10:52:34.153957+0000","last_undegraded":"2024-04-25T10:52:34.153957+0000","last_fullsized":"2024-04-25T10:52:34.153957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:22.011064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.715052+0000","last_change":"2024-04-25T10:52:32.458874+0000","last_active":"2024-04-25T10:52:33.715052+0000","last_peered":"2024-04-25T10:52:33.715052+0000","last_clean":"2024-04-25T10:52:33.715052+0000","last_became_active":"2024-04-25T10:52:32.458517+0000","last_became_peered":"2024-04-25T10:52:32.458517+0000","last_unstale":"2024-04-25T10:52:33.715052+0000","last_undegraded":"2024-04-25T10:52:33.715052+0000","last_fullsized":"2024-04-25T10:52:33.715052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:12.634521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.153846+0000","last_change":"2024-04-25T10:52:32.473068+0000","last_active":"2024-04-25T10:52:34.153846+0000","last_peered":"2024-04-25T10:52:34.153846+0000","last_clean":"2024-04-25T10:52:34.153846+0000","last_became_active":"2024-04-25T10:52:32.472713+0000","last_became_peered":"2024-04-25T10:52:32.472713+0000","last_unstale":"2024-04-25T10:52:34.153846+0000","last_undegraded":"2024-04-25T10:52:34.153846+0000","last_fullsized":"2024-04-25T10:52:34.153846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:43.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:33.791929+0000","last_change":"2024-04-25T10:52:32.468492+0000","last_active":"2024-04-25T10:52:33.791929+0000","last_peered":"2024-04-25T10:52:33.791929+0000","last_clean":"2024-04-25T10:52:33.791929+0000","last_became_active":"2024-04-25T10:52:32.468155+0000","last_became_peered":"2024-04-25T10:52:32.468155+0000","last_unstale":"2024-04-25T10:52:33.791929+0000","last_undegraded":"2024-04-25T10:52:33.791929+0000","last_fullsized":"2024-04-25T10:52:33.791929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.934230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:52:34.154824+0000","last_change":"2024-04-25T10:52:32.463057+0000","last_active":"2024-04-25T10:52:34.154824+0000","last_peered":"2024-04-25T10:52:34.154824+0000","last_clean":"2024-04-25T10:52:34.154824+0000","last_became_active":"2024-04-25T10:52:32.460983+0000","last_became_peered":"2024-04-25T10:52:32.460983+0000","last_unstale":"2024-04-25T10:52:34.154824+0000","last_undegraded":"2024-04-25T10:52:34.154824+0000","last_fullsized":"2024-04-25T10:52:34.154824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":7230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645325,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":819200,"data_stored":394047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841554,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712568,"statfs":{"total":95995035648,"available":95961669632,"internally_reserved":0,"allocated":1437696,"data_stored":986221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":394629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":827392,"data_stored":394651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1417216,"data_stored":984997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593767,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":819200,"data_stored":394047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":831488,"data_stored":395659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":394137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":395693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":827392,"data_stored":394151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":823296,"data_stored":394069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789996,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1425408,"data_stored":984597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:52:40.061 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:52:40.062 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T10:52:40.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T10:52:40.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:39 smithi112 ceph-mon[32799]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:39 smithi112 ceph-mon[32799]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 811 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T10:52:40.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:39 smithi112 ceph-mon[32799]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:39 smithi112 ceph-mon[32799]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:39 smithi112 ceph-mon[32799]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T10:52:40.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:39 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1403023294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:40.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:39 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/257813967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:40.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:39 smithi169 ceph-mon[33049]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:39 smithi169 ceph-mon[33049]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 811 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T10:52:40.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:39 smithi169 ceph-mon[33049]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:39 smithi169 ceph-mon[33049]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:52:40.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:39 smithi169 ceph-mon[33049]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T10:52:40.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:39 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1403023294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:52:40.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:39 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/257813967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:40.340 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:41.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:40 smithi112 ceph-mon[32799]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:52:41.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:40 smithi019 ceph-mon[29209]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:52:41.164 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:40 smithi169 ceph-mon[33049]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:52:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 -- 172.21.15.19:0/1577683027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4104d10 msgr2=0x7f02c4107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/1577683027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4104d10 0x7f02c4107130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009990 tx=0x7f02b4009ca0 comp rx=0 tx=0).stop 2024-04-25T10:52:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 -- 172.21.15.19:0/1577683027 shutdown_connections 2024-04-25T10:52:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/1577683027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4109fd0 0x7f02c410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/1577683027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4104d10 0x7f02c4107130 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/1577683027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02c4107670 0x7f02c4109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 -- 172.21.15.19:0/1577683027 >> 172.21.15.19:0/1577683027 conn(0x7f02c40fe750 msgr2=0x7f02c4100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:41.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:41 smithi019 ceph-mon[29209]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T10:52:41.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:41 smithi019 ceph-mon[29209]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.914+0000 7f02ca1f2700 1 -- 172.21.15.19:0/1577683027 shutdown_connections 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.915+0000 7f02ca1f2700 1 -- 172.21.15.19:0/1577683027 wait complete. 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.915+0000 7f02ca1f2700 1 Processor -- start 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.915+0000 7f02ca1f2700 1 -- start start 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02ca1f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4104d10 0x7f02c41985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02ca1f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4107670 0x7f02c4198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02ca1f2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02c4109fd0 0x7f02c41a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02ca1f2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410fc50 con 0x7f02c4107670 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02ca1f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410fad0 con 0x7f02c4104d10 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02ca1f2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410fdd0 con 0x7f02c4109fd0 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4107670 0x7f02c4198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:41.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4107670 0x7f02c4198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41360/0 (socket says 172.21.15.19:41360) 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 -- 172.21.15.19:0/868310050 learned_addr learned my addr 172.21.15.19:0/868310050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c3fff700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02c4109fd0 0x7f02c41a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c37fe700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4104d10 0x7f02c41985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 -- 172.21.15.19:0/868310050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02c4109fd0 msgr2=0x7f02c41a82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02c4109fd0 0x7f02c41a82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 -- 172.21.15.19:0/868310050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4104d10 msgr2=0x7f02c41985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4104d10 0x7f02c41985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 -- 172.21.15.19:0/868310050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c41a8950 con 0x7f02c4107670 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.916+0000 7f02c2ffd700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4107670 0x7f02c4198b20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02b800d950 tx=0x7f02b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.917+0000 7f02c0ff9700 1 -- 172.21.15.19:0/868310050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8016070 con 0x7f02c4107670 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.917+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c41a8c40 con 0x7f02c4107670 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.917+0000 7f02c0ff9700 1 -- 172.21.15.19:0/868310050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02b800fd40 con 0x7f02c4107670 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.917+0000 7f02c0ff9700 1 -- 172.21.15.19:0/868310050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b800e3b0 con 0x7f02c4107670 2024-04-25T10:52:41.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.917+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c41a9180 con 0x7f02c4107670 2024-04-25T10:52:41.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.918+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c4051190 con 0x7f02c4107670 2024-04-25T10:52:41.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.919+0000 7f02c0ff9700 1 -- 172.21.15.19:0/868310050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f02b8019070 con 0x7f02c4107670 2024-04-25T10:52:41.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.920+0000 7f02c0ff9700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f02ac0afc10 0x7f02ac0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:41.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.920+0000 7f02c37fe700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f02ac0afc10 0x7f02ac0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:41.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.920+0000 7f02c0ff9700 1 -- 172.21.15.19:0/868310050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f02b80deb10 con 0x7f02c4107670 2024-04-25T10:52:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.921+0000 7f02c37fe700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f02ac0afc10 0x7f02ac0b2120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02b400b5c0 tx=0x7f02b401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:41.924+0000 7f02c0ff9700 1 -- 172.21.15.19:0/868310050 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02b80a7640 con 0x7f02c4107670 2024-04-25T10:52:42.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:41 smithi112 ceph-mon[32799]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T10:52:42.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:41 smithi112 ceph-mon[32799]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:52:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.228+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f02c41a9350 con 0x7f02c4107670 2024-04-25T10:52:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.231+0000 7f02c0ff9700 1 -- 172.21.15.19:0/868310050 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f02b80a6f40 con 0x7f02c4107670 2024-04-25T10:52:42.231 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:52:42.231 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD6NSpmvxetDRAAUFXcr6KU8yZorhqPJN6wSg== 2024-04-25T10:52:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f02ac0afc10 msgr2=0x7f02ac0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f02ac0afc10 0x7f02ac0b2120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02b400b5c0 tx=0x7f02b401a040 comp rx=0 tx=0).stop 2024-04-25T10:52:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4107670 msgr2=0x7f02c4198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4107670 0x7f02c4198b20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02b800d950 tx=0x7f02b800dc60 comp rx=0 tx=0).stop 2024-04-25T10:52:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 shutdown_connections 2024-04-25T10:52:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f02ac0afc10 0x7f02ac0b2120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02c4109fd0 0x7f02c41a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4104d10 0x7f02c41985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 --2- 172.21.15.19:0/868310050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02c4107670 0x7f02c4198b20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 >> 172.21.15.19:0/868310050 conn(0x7f02c40fe750 msgr2=0x7f02c4103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 shutdown_connections 2024-04-25T10:52:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:42.233+0000 7f02ca1f2700 1 -- 172.21.15.19:0/868310050 wait complete. 2024-04-25T10:52:42.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:41 smithi169 ceph-mon[33049]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T10:52:42.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:41 smithi169 ceph-mon[33049]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:52:42.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:42 smithi019 ceph-mon[29209]: pgmap v283: 97 pgs: 97 active+clean; 615 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-25T10:52:42.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:42 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/868310050' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T10:52:42.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:42 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/868310050' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T10:52:42.880 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:52:42.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T10:52:42.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:52:42.919 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:52:42.919 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:52:42.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T10:52:43.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:42 smithi112 ceph-mon[32799]: pgmap v283: 97 pgs: 97 active+clean; 615 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-25T10:52:43.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:42 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/868310050' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T10:52:43.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:42 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/868310050' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T10:52:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:43.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:42 smithi169 ceph-mon[33049]: pgmap v283: 97 pgs: 97 active+clean; 615 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-25T10:52:43.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:42 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/868310050' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T10:52:43.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:42 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/868310050' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T10:52:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.785+0000 7fd6d804d700 1 -- 172.21.15.19:0/2021796883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0107c90 msgr2=0x7fd6d01080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.785+0000 7fd6d804d700 1 --2- 172.21.15.19:0/2021796883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0107c90 0x7fd6d01080a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40099c0 tx=0x7fd6c4009cd0 comp rx=0 tx=0).stop 2024-04-25T10:52:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.785+0000 7fd6d804d700 1 -- 172.21.15.19:0/2021796883 shutdown_connections 2024-04-25T10:52:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.785+0000 7fd6d804d700 1 --2- 172.21.15.19:0/2021796883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d01085e0 0x7fd6d010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.785+0000 7fd6d804d700 1 --2- 172.21.15.19:0/2021796883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d0103360 0x7fd6d0103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.785+0000 7fd6d804d700 1 --2- 172.21.15.19:0/2021796883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0107c90 0x7fd6d01080a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.785+0000 7fd6d804d700 1 -- 172.21.15.19:0/2021796883 >> 172.21.15.19:0/2021796883 conn(0x7fd6d00fecb0 msgr2=0x7fd6d01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.786+0000 7fd6d804d700 1 -- 172.21.15.19:0/2021796883 shutdown_connections 2024-04-25T10:52:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.786+0000 7fd6d804d700 1 -- 172.21.15.19:0/2021796883 wait complete. 2024-04-25T10:52:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.786+0000 7fd6d804d700 1 Processor -- start 2024-04-25T10:52:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d804d700 1 -- start start 2024-04-25T10:52:44.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d804d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0103360 0x7fd6d019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:44.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d804d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d0107c90 0x7fd6d019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d804d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d01085e0 0x7fd6d019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d804d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01131f0 con 0x7fd6d0103360 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d804d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0113070 con 0x7fd6d0107c90 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d804d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0113370 con 0x7fd6d01085e0 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0103360 0x7fd6d019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0103360 0x7fd6d019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59116/0 (socket says 172.21.15.19:59116) 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 -- 172.21.15.19:0/1085989777 learned_addr learned my addr 172.21.15.19:0/1085989777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d55e8700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d0107c90 0x7fd6d019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d65ea700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d01085e0 0x7fd6d019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 -- 172.21.15.19:0/1085989777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d01085e0 msgr2=0x7fd6d019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d01085e0 0x7fd6d019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 -- 172.21.15.19:0/1085989777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d0107c90 msgr2=0x7fd6d019c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d0107c90 0x7fd6d019c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.787+0000 7fd6d5de9700 1 -- 172.21.15.19:0/1085989777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01a6e40 con 0x7fd6d0103360 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.788+0000 7fd6d5de9700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0103360 0x7fd6d019baf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8007f10 tx=0x7fd6c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.788+0000 7fd6ceffd700 1 -- 172.21.15.19:0/1085989777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800ddd0 con 0x7fd6d0103360 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.788+0000 7fd6ceffd700 1 -- 172.21.15.19:0/1085989777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8004bb0 con 0x7fd6d0103360 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.788+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01a7130 con 0x7fd6d0103360 2024-04-25T10:52:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.788+0000 7fd6ceffd700 1 -- 172.21.15.19:0/1085989777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80046a0 con 0x7fd6d0103360 2024-04-25T10:52:44.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.788+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a7690 con 0x7fd6d0103360 2024-04-25T10:52:44.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.789+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0051190 con 0x7fd6d0103360 2024-04-25T10:52:44.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.790+0000 7fd6ceffd700 1 -- 172.21.15.19:0/1085989777 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd6c801c020 con 0x7fd6d0103360 2024-04-25T10:52:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.791+0000 7fd6ceffd700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6b80afc90 0x7fd6b80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.791+0000 7fd6d55e8700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6b80afc90 0x7fd6b80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.791+0000 7fd6ceffd700 1 -- 172.21.15.19:0/1085989777 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd6c80def40 con 0x7fd6d0103360 2024-04-25T10:52:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.792+0000 7fd6d55e8700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6b80afc90 0x7fd6b80b21a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4006010 tx=0x7fd6c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:44.795+0000 7fd6ceffd700 1 -- 172.21.15.19:0/1085989777 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6c8023030 con 0x7fd6d0103360 2024-04-25T10:52:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.066+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd6d002e510 con 0x7fd6d0103360 2024-04-25T10:52:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.071+0000 7fd6ceffd700 1 -- 172.21.15.19:0/1085989777 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284478 (secure 0 0 0) 0x7fd6c8010070 con 0x7fd6d0103360 2024-04-25T10:52:45.075 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6b80afc90 msgr2=0x7fd6b80b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6b80afc90 0x7fd6b80b21a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4006010 tx=0x7fd6c4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:52:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0103360 msgr2=0x7fd6d019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0103360 0x7fd6d019baf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8007f10 tx=0x7fd6c800d3b0 comp rx=0 tx=0).stop 2024-04-25T10:52:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 shutdown_connections 2024-04-25T10:52:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6b80afc90 0x7fd6b80b21a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d01085e0 0x7fd6d019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6d0103360 0x7fd6d019baf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 --2- 172.21.15.19:0/1085989777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d0107c90 0x7fd6d019c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 >> 172.21.15.19:0/1085989777 conn(0x7fd6d00fecb0 msgr2=0x7fd6d0109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 shutdown_connections 2024-04-25T10:52:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:45.074+0000 7fd6d804d700 1 -- 172.21.15.19:0/1085989777 wait complete. 2024-04-25T10:52:45.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:45 smithi112 ceph-mon[32799]: pgmap v284: 97 pgs: 97 active+clean; 615 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.2 KiB/s wr, 5 op/s 2024-04-25T10:52:45.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:45 smithi019 ceph-mon[29209]: pgmap v284: 97 pgs: 97 active+clean; 615 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.2 KiB/s wr, 5 op/s 2024-04-25T10:52:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:45 smithi169 ceph-mon[33049]: pgmap v284: 97 pgs: 97 active+clean; 615 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.2 KiB/s wr, 5 op/s 2024-04-25T10:52:45.700 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1708221429},{"type":"v1","addr":"172.21.15.19:6801","nonce":1708221429}]},"active_addr":"172.21.15.19:6801/1708221429","active_change":"2024-04-25T10:45:29.902566+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2535542967}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3805408646}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":48031724}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":4264937365}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3245735226}]}]} 2024-04-25T10:52:45.706 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T10:52:45.706 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T10:52:45.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T10:52:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:46.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:46 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1085989777' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T10:52:46.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:46 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1085989777' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T10:52:46.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:46 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1085989777' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T10:52:47.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:47 smithi019 ceph-mon[29209]: pgmap v285: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-04-25T10:52:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:47 smithi169 ceph-mon[33049]: pgmap v285: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-04-25T10:52:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:47 smithi112 ceph-mon[32799]: pgmap v285: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-04-25T10:52:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3963200003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c108650 msgr2=0x7f5b5c10fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3963200003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c108650 0x7f5b5c10fff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400b0d0 tx=0x7f5b5400b3e0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3963200003 shutdown_connections 2024-04-25T10:52:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3963200003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c108650 0x7f5b5c10fff0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3963200003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b5c1039e0 0x7f5b5c103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3963200003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b5c107d00 0x7f5b5c108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3963200003 >> 172.21.15.19:0/3963200003 conn(0x7f5b5c0fe370 msgr2=0x7f5b5c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:47.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3963200003 shutdown_connections 2024-04-25T10:52:47.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.708+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3963200003 wait complete. 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 Processor -- start 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 -- start start 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1039e0 0x7f5b5c19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b5c107d00 0x7f5b5c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b5c108650 0x7f5b5c1a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1132c0 con 0x7f5b5c1039e0 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c113140 con 0x7f5b5c108650 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b60b6f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c113440 con 0x7f5b5c107d00 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b5a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1039e0 0x7f5b5c19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b5a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1039e0 0x7f5b5c19baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59140/0 (socket says 172.21.15.19:59140) 2024-04-25T10:52:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.709+0000 7f5b5a59c700 1 -- 172.21.15.19:0/3985926841 learned_addr learned my addr 172.21.15.19:0/3985926841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5ad9d700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b5c108650 0x7f5b5c1a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b59d9b700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b5c107d00 0x7f5b5c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5a59c700 1 -- 172.21.15.19:0/3985926841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b5c107d00 msgr2=0x7f5b5c19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5a59c700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b5c107d00 0x7f5b5c19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5a59c700 1 -- 172.21.15.19:0/3985926841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b5c108650 msgr2=0x7f5b5c1a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5a59c700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b5c108650 0x7f5b5c1a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5a59c700 1 -- 172.21.15.19:0/3985926841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c19c890 con 0x7f5b5c1039e0 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5ad9d700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b5c108650 0x7f5b5c1a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b5a59c700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1039e0 0x7f5b5c19baa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00d950 tx=0x7f5b4c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b4b7fe700 1 -- 172.21.15.19:0/3985926841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c016070 con 0x7f5b5c1039e0 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c1a7180 con 0x7f5b5c1039e0 2024-04-25T10:52:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b4b7fe700 1 -- 172.21.15.19:0/3985926841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b4c010460 con 0x7f5b5c1039e0 2024-04-25T10:52:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b4b7fe700 1 -- 172.21.15.19:0/3985926841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00f620 con 0x7f5b5c1039e0 2024-04-25T10:52:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.710+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c1a7690 con 0x7f5b5c1039e0 2024-04-25T10:52:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.711+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5c051190 con 0x7f5b5c1039e0 2024-04-25T10:52:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.713+0000 7f5b4b7fe700 1 -- 172.21.15.19:0/3985926841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5b4c019070 con 0x7f5b5c1039e0 2024-04-25T10:52:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.713+0000 7f5b4b7fe700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5b440afc10 0x7f5b440b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.713+0000 7f5b59d9b700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5b440afc10 0x7f5b440b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.714+0000 7f5b4b7fe700 1 -- 172.21.15.19:0/3985926841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f5b4c0de850 con 0x7f5b5c1039e0 2024-04-25T10:52:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.714+0000 7f5b59d9b700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5b440afc10 0x7f5b440b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b50005950 tx=0x7f5b500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:47.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.717+0000 7f5b4b7fe700 1 -- 172.21.15.19:0/3985926841 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b4c0a7480 con 0x7f5b5c1039e0 2024-04-25T10:52:47.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.925+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b5c02e510 con 0x7f5b5c1039e0 2024-04-25T10:52:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.927+0000 7f5b4b7fe700 1 -- 172.21.15.19:0/3985926841 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22226 (secure 0 0 0) 0x7f5b4c0a6d80 con 0x7f5b5c1039e0 2024-04-25T10:52:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:47.928 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":77,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:52:33.451613+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:52:47.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5b440afc10 msgr2=0x7f5b440b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:47.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5b440afc10 0x7f5b440b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b50005950 tx=0x7f5b500058e0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1039e0 msgr2=0x7f5b5c19baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:47.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1039e0 0x7f5b5c19baa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00d950 tx=0x7f5b4c00dd10 comp rx=0 tx=0).stop 2024-04-25T10:52:47.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 shutdown_connections 2024-04-25T10:52:47.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f5b440afc10 0x7f5b440b2120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b5c108650 0x7f5b5c1a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1039e0 0x7f5b5c19baa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 --2- 172.21.15.19:0/3985926841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b5c107d00 0x7f5b5c19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:47.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 >> 172.21.15.19:0/3985926841 conn(0x7f5b5c0fe370 msgr2=0x7f5b5c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:47.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 shutdown_connections 2024-04-25T10:52:47.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:47.929+0000 7f5b60b6f700 1 -- 172.21.15.19:0/3985926841 wait complete. 2024-04-25T10:52:48.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:48 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3985926841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:48.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:48 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3985926841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:48.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:48 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3985926841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:48.616 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T10:52:48.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T10:52:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:49.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:49 smithi019 ceph-mon[29209]: pgmap v286: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2024-04-25T10:52:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:49 smithi169 ceph-mon[33049]: pgmap v286: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2024-04-25T10:52:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:49 smithi112 ceph-mon[32799]: pgmap v286: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2024-04-25T10:52:50.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3011451044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 msgr2=0x7ff8d8110c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3011451044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d8110c60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800b0a0 tx=0x7ff8c800b3b0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3011451044 shutdown_connections 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3011451044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d8110c60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3011451044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d81053c0 0x7ff8d81057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3011451044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d8105d10 0x7ff8d810e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3011451044 >> 172.21.15.19:0/3011451044 conn(0x7ff8d80fecb0 msgr2=0x7ff8d81010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3011451044 shutdown_connections 2024-04-25T10:52:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.757+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3011451044 wait complete. 2024-04-25T10:52:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.758+0000 7ff8dd4b3700 1 Processor -- start 2024-04-25T10:52:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.758+0000 7ff8dd4b3700 1 -- start start 2024-04-25T10:52:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8dd4b3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d81053c0 0x7ff8d81a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8dd4b3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d8105d10 0x7ff8d81a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8dd4b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8dd4b3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d81142e0 con 0x7ff8d810e7a0 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8dd4b3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8114160 con 0x7ff8d81053c0 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8dd4b3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8114460 con 0x7ff8d8105d10 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59156/0 (socket says 172.21.15.19:59156) 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d67fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d8105d10 0x7ff8d81a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d77fe700 1 -- 172.21.15.19:0/3860562874 learned_addr learned my addr 172.21.15.19:0/3860562874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d6ffd700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d81053c0 0x7ff8d81a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d67fc700 1 -- 172.21.15.19:0/3860562874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d81053c0 msgr2=0x7ff8d81a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d67fc700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d81053c0 0x7ff8d81a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d67fc700 1 -- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 msgr2=0x7ff8d819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d67fc700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d67fc700 1 -- 172.21.15.19:0/3860562874 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d819b0f0 con 0x7ff8d8105d10 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d77fe700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.759+0000 7ff8d67fc700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d8105d10 0x7ff8d81a0d60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000d950 tx=0x7ff8c000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.760+0000 7ff8cffff700 1 -- 172.21.15.19:0/3860562874 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0016070 con 0x7ff8d8105d10 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.760+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d819b3e0 con 0x7ff8d8105d10 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.760+0000 7ff8cffff700 1 -- 172.21.15.19:0/3860562874 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c000fd40 con 0x7ff8d8105d10 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.760+0000 7ff8cffff700 1 -- 172.21.15.19:0/3860562874 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000e3b0 con 0x7ff8d8105d10 2024-04-25T10:52:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.760+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d81a76b0 con 0x7ff8d8105d10 2024-04-25T10:52:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.761+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d8051190 con 0x7ff8d8105d10 2024-04-25T10:52:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.763+0000 7ff8cffff700 1 -- 172.21.15.19:0/3860562874 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ff8c0019070 con 0x7ff8d8105d10 2024-04-25T10:52:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.763+0000 7ff8cffff700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8c40afc10 0x7ff8c40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.763+0000 7ff8d6ffd700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8c40afc10 0x7ff8c40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.763+0000 7ff8cffff700 1 -- 172.21.15.19:0/3860562874 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7ff8c00debb0 con 0x7ff8d8105d10 2024-04-25T10:52:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.764+0000 7ff8d6ffd700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8c40afc10 0x7ff8c40b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8d819bb80 tx=0x7ff8d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.767+0000 7ff8cffff700 1 -- 172.21.15.19:0/3860562874 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8c00a76e0 con 0x7ff8d8105d10 2024-04-25T10:52:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.974+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8d819bf70 con 0x7ff8d8105d10 2024-04-25T10:52:50.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.976+0000 7ff8cffff700 1 -- 172.21.15.19:0/3860562874 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22226 (secure 0 0 0) 0x7ff8c00a6fe0 con 0x7ff8d8105d10 2024-04-25T10:52:50.977 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:52:50.977 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":77,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:52:33.451613+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:52:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8c40afc10 msgr2=0x7ff8c40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8c40afc10 0x7ff8c40b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8d819bb80 tx=0x7ff8d0009260 comp rx=0 tx=0).stop 2024-04-25T10:52:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d8105d10 msgr2=0x7ff8d81a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d8105d10 0x7ff8d81a0d60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000d950 tx=0x7ff8c000dc60 comp rx=0 tx=0).stop 2024-04-25T10:52:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 shutdown_connections 2024-04-25T10:52:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff8c40afc10 0x7ff8c40b2120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8d810e7a0 0x7ff8d819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d81053c0 0x7ff8d81a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 --2- 172.21.15.19:0/3860562874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d8105d10 0x7ff8d81a0d60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.978+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 >> 172.21.15.19:0/3860562874 conn(0x7ff8d80fecb0 msgr2=0x7ff8d8100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.979+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 shutdown_connections 2024-04-25T10:52:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:50.979+0000 7ff8dd4b3700 1 -- 172.21.15.19:0/3860562874 wait complete. 2024-04-25T10:52:51.326 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:51 smithi019 ceph-mon[29209]: pgmap v287: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-25T10:52:51.326 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:51 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3860562874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:51 smithi169 ceph-mon[33049]: pgmap v287: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-25T10:52:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:51 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3860562874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:51.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:51 smithi112 ceph-mon[32799]: pgmap v287: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-25T10:52:51.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:51 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3860562874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:52:51.650 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T10:52:51.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T10:52:51.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T10:52:51.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T10:52:51.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T10:52:51.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T10:52:51.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T10:52:51.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T10:52:51.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T10:52:51.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T10:52:51.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T10:52:51.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T10:52:52.257 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:52.258 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:53.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:53 smithi019 ceph-mon[29209]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T10:52:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:53 smithi169 ceph-mon[33049]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T10:52:53.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:53 smithi112 ceph-mon[32799]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T10:52:54.050 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.051 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.967 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.967 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:55.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:55 smithi019 ceph-mon[29209]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T10:52:55.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 -- 172.21.15.19:0/4003407970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90103360 msgr2=0x7f1a90103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:55.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 --2- 172.21.15.19:0/4003407970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90103360 0x7f1a90103770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009990 tx=0x7f1a8c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 -- 172.21.15.19:0/4003407970 shutdown_connections 2024-04-25T10:52:55.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 --2- 172.21.15.19:0/4003407970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a901085e0 0x7f1a9010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 --2- 172.21.15.19:0/4003407970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90103360 0x7f1a90103770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 --2- 172.21.15.19:0/4003407970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a90107c90 0x7f1a901080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 -- 172.21.15.19:0/4003407970 >> 172.21.15.19:0/4003407970 conn(0x7f1a900fec90 msgr2=0x7f1a901010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:55.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 -- 172.21.15.19:0/4003407970 shutdown_connections 2024-04-25T10:52:55.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.451+0000 7f1a98774700 1 -- 172.21.15.19:0/4003407970 wait complete. 2024-04-25T10:52:55.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.452+0000 7f1a98774700 1 Processor -- start 2024-04-25T10:52:55.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.452+0000 7f1a98774700 1 -- start start 2024-04-25T10:52:55.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a98774700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90103360 0x7f1a901a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:55.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a98774700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90107c90 0x7f1a901a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a98774700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a901085e0 0x7f1a9019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a95d0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90107c90 0x7f1a901a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a98774700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a90114370 con 0x7f1a90107c90 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a95d0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90107c90 0x7f1a901a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43736/0 (socket says 172.21.15.19:43736) 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a95d0f700 1 -- 172.21.15.19:0/1537456738 learned_addr learned my addr 172.21.15.19:0/1537456738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901141f0 con 0x7f1a901085e0 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901144f0 con 0x7f1a90103360 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a96510700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90103360 0x7f1a901a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.453+0000 7f1a96d11700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a901085e0 0x7f1a9019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a95d0f700 1 -- 172.21.15.19:0/1537456738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90103360 msgr2=0x7f1a901a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a95d0f700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90103360 0x7f1a901a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a95d0f700 1 -- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a901085e0 msgr2=0x7f1a9019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a95d0f700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a901085e0 0x7f1a9019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a95d0f700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9019b160 con 0x7f1a90107c90 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a95d0f700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90107c90 0x7f1a901a0dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009d10 tx=0x7f1a8000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a96d11700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a901085e0 0x7f1a9019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80009e00 con 0x7f1a90107c90 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a8000bdc0 con 0x7f1a90107c90 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80019410 con 0x7f1a90107c90 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.454+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9019b450 con 0x7f1a90107c90 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.455+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a90110480 con 0x7f1a90107c90 2024-04-25T10:52:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.456+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a90063e20 con 0x7f1a90107c90 2024-04-25T10:52:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.458+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1a80004750 con 0x7f1a90107c90 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.459+0000 7f1a877fe700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a7c0afc90 0x7f1a7c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.459+0000 7f1a96510700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a7c0afc90 0x7f1a7c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.459+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f1a800e3060 con 0x7f1a90107c90 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.459+0000 7f1a877fe700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] conn(0x7f1a7c0b94c0 0x7f1a7c0bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.459+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a8001aec0 con 0x7f1a7c0b94c0 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.459+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1a800e3540 con 0x7f1a90107c90 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.459+0000 7f1a96510700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a7c0afc90 0x7f1a7c0b21a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00b2e0 tx=0x7f1a8c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.460+0000 7f1a96d11700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] conn(0x7f1a7c0b94c0 0x7f1a7c0bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.460+0000 7f1a96d11700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] conn(0x7f1a7c0b94c0 0x7f1a7c0bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.461+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== osd.4 v2:172.21.15.112:6808/3068928119 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a8001aec0 con 0x7f1a7c0b94c0 2024-04-25T10:52:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.477+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 --> [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a90064220 con 0x7f1a7c0b94c0 2024-04-25T10:52:55.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a877fe700 1 -- 172.21.15.19:0/1537456738 <== osd.4 v2:172.21.15.112:6808/3068928119 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a90064220 con 0x7f1a7c0b94c0 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] conn(0x7f1a7c0b94c0 msgr2=0x7f1a7c0bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] conn(0x7f1a7c0b94c0 0x7f1a7c0bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a7c0afc90 msgr2=0x7f1a7c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1a7c0afc90 0x7f1a7c0b21a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00b2e0 tx=0x7f1a8c005960 comp rx=0 tx=0).stop 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90107c90 msgr2=0x7f1a901a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 --2- 172.21.15.19:0/1537456738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a90107c90 0x7f1a901a0dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009d10 tx=0x7f1a8000b5a0 comp rx=0 tx=0).stop 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a96d11700 1 -- 172.21.15.19:0/1537456738 reap_dead start 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 shutdown_connections 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.478+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 >> 172.21.15.19:0/1537456738 conn(0x7f1a900fec90 msgr2=0x7f1a90107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.479+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 shutdown_connections 2024-04-25T10:52:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:55.479+0000 7f1a98774700 1 -- 172.21.15.19:0/1537456738 wait complete. 2024-04-25T10:52:55.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:55 smithi169 ceph-mon[33049]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T10:52:55.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:55 smithi112 ceph-mon[32799]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T10:52:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.597+0000 7f9b83f16700 1 -- 172.21.15.19:0/4055624358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 msgr2=0x7f9b7c109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.597+0000 7f9b83f16700 1 --2- 172.21.15.19:0/4055624358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 0x7f9b7c109200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b780099c0 tx=0x7f9b78009cd0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.597+0000 7f9b83f16700 1 -- 172.21.15.19:0/4055624358 shutdown_connections 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.597+0000 7f9b83f16700 1 --2- 172.21.15.19:0/4055624358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b7c109740 0x7f9b7c1110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.597+0000 7f9b83f16700 1 --2- 172.21.15.19:0/4055624358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c1029d0 0x7f9b7c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.597+0000 7f9b83f16700 1 --2- 172.21.15.19:0/4055624358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 0x7f9b7c109200 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.597+0000 7f9b83f16700 1 -- 172.21.15.19:0/4055624358 >> 172.21.15.19:0/4055624358 conn(0x7f9b7c0fe370 msgr2=0x7f9b7c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.598+0000 7f9b83f16700 1 -- 172.21.15.19:0/4055624358 shutdown_connections 2024-04-25T10:52:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.598+0000 7f9b83f16700 1 -- 172.21.15.19:0/4055624358 wait complete. 2024-04-25T10:52:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.598+0000 7f9b83f16700 1 Processor -- start 2024-04-25T10:52:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.598+0000 7f9b83f16700 1 -- start start 2024-04-25T10:52:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b83f16700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b7c1029d0 0x7f9b7c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b83f16700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 0x7f9b7c1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b83f16700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c109740 0x7f9b7c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b83f16700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c114390 con 0x7f9b7c108df0 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b83f16700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c114210 con 0x7f9b7c1029d0 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b83f16700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c114510 con 0x7f9b7c109740 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 0x7f9b7c1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 0x7f9b7c1a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43750/0 (socket says 172.21.15.19:43750) 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 -- 172.21.15.19:0/847257355 learned_addr learned my addr 172.21.15.19:0/847257355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b81cb2700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b7c1029d0 0x7f9b7c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b824b3700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c109740 0x7f9b7c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 -- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c109740 msgr2=0x7f9b7c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c109740 0x7f9b7c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 -- 172.21.15.19:0/847257355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b7c1029d0 msgr2=0x7f9b7c1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b7c1029d0 0x7f9b7c1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 -- 172.21.15.19:0/847257355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c19b1f0 con 0x7f9b7c108df0 2024-04-25T10:52:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b814b1700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 0x7f9b7c1a0f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009f50 tx=0x7f9b780037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.599+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78027070 con 0x7f9b7c108df0 2024-04-25T10:52:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.600+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b78004090 con 0x7f9b7c108df0 2024-04-25T10:52:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.600+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c19b480 con 0x7f9b7c108df0 2024-04-25T10:52:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.600+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78022e40 con 0x7f9b7c108df0 2024-04-25T10:52:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.600+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c19b9b0 con 0x7f9b7c108df0 2024-04-25T10:52:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.600+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b7c063da0 con 0x7f9b7c108df0 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.602+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9b7802a070 con 0x7f9b7c108df0 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b72ffd700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f9b680afc90 0x7f9b680b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b81cb2700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f9b680afc90 0x7f9b680b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f9b780e0100 con 0x7f9b7c108df0 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b72ffd700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] conn(0x7f9b680b94c0 0x7f9b680bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 --> [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b7800eea0 con 0x7f9b680b94c0 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f9b780e04b0 con 0x7f9b7c108df0 2024-04-25T10:52:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b81cb2700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f9b680afc90 0x7f9b680b21a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c06bb00 tx=0x7f9b6c00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.603+0000 7f9b824b3700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] conn(0x7f9b680b94c0 0x7f9b680bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:52:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.604+0000 7f9b824b3700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] conn(0x7f9b680b94c0 0x7f9b680bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:52:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.605+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== osd.5 v2:172.21.15.169:6808/3010722907 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b7800eea0 con 0x7f9b680b94c0 2024-04-25T10:52:56.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.621+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 --> [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b7c0641a0 con 0x7f9b680b94c0 2024-04-25T10:52:56.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b72ffd700 1 -- 172.21.15.19:0/847257355 <== osd.5 v2:172.21.15.169:6808/3010722907 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b7c0641a0 con 0x7f9b680b94c0 2024-04-25T10:52:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] conn(0x7f9b680b94c0 msgr2=0x7f9b680bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] conn(0x7f9b680b94c0 0x7f9b680bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f9b680afc90 msgr2=0x7f9b680b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f9b680afc90 0x7f9b680b21a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c06bb00 tx=0x7f9b6c00a3b0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 msgr2=0x7f9b7c1a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:52:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 --2- 172.21.15.19:0/847257355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7c108df0 0x7f9b7c1a0f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009f50 tx=0x7f9b780037c0 comp rx=0 tx=0).stop 2024-04-25T10:52:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b824b3700 1 -- 172.21.15.19:0/847257355 reap_dead start 2024-04-25T10:52:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 shutdown_connections 2024-04-25T10:52:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 >> 172.21.15.19:0/847257355 conn(0x7f9b7c0fe370 msgr2=0x7f9b7c0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:52:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.622+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 shutdown_connections 2024-04-25T10:52:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:52:56.623+0000 7f9b83f16700 1 -- 172.21.15.19:0/847257355 wait complete. 2024-04-25T10:52:57.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:57 smithi019 ceph-mon[29209]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-25T10:52:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:57 smithi169 ceph-mon[33049]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-25T10:52:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:57 smithi112 ceph-mon[32799]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-25T10:52:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:52:59.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:52:59 smithi019 ceph-mon[29209]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T10:52:59.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:52:59 smithi169 ceph-mon[33049]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T10:52:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:52:59 smithi112 ceph-mon[32799]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T10:53:01.333 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:01 smithi019 ceph-mon[29209]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T10:53:01.335 INFO:teuthology.orchestra.run.smithi019.stdout:158913790001 2024-04-25T10:53:01.335 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T10:53:01.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:01 smithi169 ceph-mon[33049]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T10:53:01.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:01 smithi112 ceph-mon[32799]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T10:53:03.033 INFO:teuthology.orchestra.run.smithi019.stdout:137438953525 2024-04-25T10:53:03.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T10:53:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:03 smithi019 ceph-mon[29209]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T10:53:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:03 smithi169 ceph-mon[33049]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T10:53:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:03 smithi112 ceph-mon[32799]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T10:53:03.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.729+0000 7fe18df68700 1 -- 172.21.15.19:0/2063262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1881053d0 msgr2=0x7fe1881057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.729+0000 7fe18df68700 1 --2- 172.21.15.19:0/2063262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1881053d0 0x7fe1881057e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe178009990 tx=0x7fe178009ca0 comp rx=0 tx=0).stop 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.729+0000 7fe18df68700 1 -- 172.21.15.19:0/2063262 shutdown_connections 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.729+0000 7fe18df68700 1 --2- 172.21.15.19:0/2063262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18810e7b0 0x7fe188110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.729+0000 7fe18df68700 1 --2- 172.21.15.19:0/2063262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1881053d0 0x7fe1881057e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.729+0000 7fe18df68700 1 --2- 172.21.15.19:0/2063262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe188105d20 0x7fe18810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.729+0000 7fe18df68700 1 -- 172.21.15.19:0/2063262 >> 172.21.15.19:0/2063262 conn(0x7fe1880fec90 msgr2=0x7fe1881010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.730+0000 7fe18df68700 1 -- 172.21.15.19:0/2063262 shutdown_connections 2024-04-25T10:53:03.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.730+0000 7fe18df68700 1 -- 172.21.15.19:0/2063262 wait complete. 2024-04-25T10:53:03.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.730+0000 7fe18df68700 1 Processor -- start 2024-04-25T10:53:03.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.730+0000 7fe18df68700 1 -- start start 2024-04-25T10:53:03.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe18df68700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1881053d0 0x7fe18819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:03.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe18df68700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe188105d20 0x7fe18819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:03.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe18df68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18810e7b0 0x7fe18819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe18df68700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881143d0 con 0x7fe18810e7b0 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe18df68700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe188114250 con 0x7fe188105d20 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe18df68700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe188114550 con 0x7fe1881053d0 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18810e7b0 0x7fe18819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18810e7b0 0x7fe18819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59126/0 (socket says 172.21.15.19:59126) 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 -- 172.21.15.19:0/2656592168 learned_addr learned my addr 172.21.15.19:0/2656592168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe186ffd700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe188105d20 0x7fe18819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe1877fe700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1881053d0 0x7fe18819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 -- 172.21.15.19:0/2656592168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1881053d0 msgr2=0x7fe18819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1881053d0 0x7fe18819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 -- 172.21.15.19:0/2656592168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe188105d20 msgr2=0x7fe18819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe188105d20 0x7fe18819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 -- 172.21.15.19:0/2656592168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18819fc10 con 0x7fe18810e7b0 2024-04-25T10:53:03.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.731+0000 7fe187fff700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18810e7b0 0x7fe18819f570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b970 tx=0x7fe18000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:03.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.732+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180019070 con 0x7fe18810e7b0 2024-04-25T10:53:03.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.732+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18000cde0 con 0x7fe18810e7b0 2024-04-25T10:53:03.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.732+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18819ff00 con 0x7fe18810e7b0 2024-04-25T10:53:03.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.732+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180007370 con 0x7fe18810e7b0 2024-04-25T10:53:03.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.732+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a7820 con 0x7fe18810e7b0 2024-04-25T10:53:03.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.733+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe188063de0 con 0x7fe18810e7b0 2024-04-25T10:53:03.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.734+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe18001c070 con 0x7fe18810e7b0 2024-04-25T10:53:03.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe184ff9700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1700afc10 0x7fe1700b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:03.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe1877fe700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1700afc10 0x7fe1700b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:03.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fe1800deda0 con 0x7fe18810e7b0 2024-04-25T10:53:03.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe184ff9700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] conn(0x7fe1700b9440 0x7fe1700bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:03.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 --> [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1700bbf60 con 0x7fe1700b9440 2024-04-25T10:53:03.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe1800df280 con 0x7fe18810e7b0 2024-04-25T10:53:03.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe186ffd700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] conn(0x7fe1700b9440 0x7fe1700bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:03.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.735+0000 7fe1877fe700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1700afc10 0x7fe1700b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe178003800 tx=0x7fe17801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:03.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.736+0000 7fe186ffd700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] conn(0x7fe1700b9440 0x7fe1700bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:03.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.736+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== osd.9 v2:172.21.15.19:6826/920660159 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1700bbf60 con 0x7fe1700b9440 2024-04-25T10:53:03.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.753+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 --> [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe1880641e0 con 0x7fe1700b9440 2024-04-25T10:53:03.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe184ff9700 1 -- 172.21.15.19:0/2656592168 <== osd.9 v2:172.21.15.19:6826/920660159 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe1880641e0 con 0x7fe1700b9440 2024-04-25T10:53:03.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] conn(0x7fe1700b9440 msgr2=0x7fe1700bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:03.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] conn(0x7fe1700b9440 0x7fe1700bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:03.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1700afc10 msgr2=0x7fe1700b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe1700afc10 0x7fe1700b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe178003800 tx=0x7fe17801a040 comp rx=0 tx=0).stop 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18810e7b0 msgr2=0x7fe18819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 --2- 172.21.15.19:0/2656592168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18810e7b0 0x7fe18819f570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b970 tx=0x7fe18000bc80 comp rx=0 tx=0).stop 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe187fff700 1 -- 172.21.15.19:0/2656592168 reap_dead start 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 shutdown_connections 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 >> 172.21.15.19:0/2656592168 conn(0x7fe1880fec90 msgr2=0x7fe1880ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 shutdown_connections 2024-04-25T10:53:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:03.754+0000 7fe18df68700 1 -- 172.21.15.19:0/2656592168 wait complete. 2024-04-25T10:53:05.174 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:05 smithi019 ceph-mon[29209]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:53:05.202 INFO:teuthology.orchestra.run.smithi019.stdout:257698037790 2024-04-25T10:53:05.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T10:53:05.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:05 smithi169 ceph-mon[33049]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:53:05.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:05 smithi112 ceph-mon[32799]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:53:07.283 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:07 smithi019 ceph-mon[29209]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:07.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 -- 172.21.15.19:0/4250556493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 msgr2=0x7fd8f41057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 --2- 172.21.15.19:0/4250556493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 0x7fd8f41057e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009990 tx=0x7fd8e4009ca0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 -- 172.21.15.19:0/4250556493 shutdown_connections 2024-04-25T10:53:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 --2- 172.21.15.19:0/4250556493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f410e7b0 0x7fd8f4110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 --2- 172.21.15.19:0/4250556493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 0x7fd8f41057e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 --2- 172.21.15.19:0/4250556493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4105d20 0x7fd8f410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 -- 172.21.15.19:0/4250556493 >> 172.21.15.19:0/4250556493 conn(0x7fd8f40fec90 msgr2=0x7fd8f41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 -- 172.21.15.19:0/4250556493 shutdown_connections 2024-04-25T10:53:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.484+0000 7fd8f9a1d700 1 -- 172.21.15.19:0/4250556493 wait complete. 2024-04-25T10:53:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 Processor -- start 2024-04-25T10:53:07.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 -- start start 2024-04-25T10:53:07.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 0x7fd8f419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4105d20 0x7fd8f419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f410e7b0 0x7fd8f419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4114340 con 0x7fd8f41053d0 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41141c0 con 0x7fd8f410e7b0 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.485+0000 7fd8f9a1d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41144c0 con 0x7fd8f4105d20 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 0x7fd8f419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 0x7fd8f419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59174/0 (socket says 172.21.15.19:59174) 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 -- 172.21.15.19:0/2913304064 learned_addr learned my addr 172.21.15.19:0/2913304064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f27fc700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4105d20 0x7fd8f419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f37fe700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f410e7b0 0x7fd8f419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 -- 172.21.15.19:0/2913304064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4105d20 msgr2=0x7fd8f419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4105d20 0x7fd8f419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 -- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f410e7b0 msgr2=0x7fd8f419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f410e7b0 0x7fd8f419f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 -- 172.21.15.19:0/2913304064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f419fbe0 con 0x7fd8f41053d0 2024-04-25T10:53:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f37fe700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f410e7b0 0x7fd8f419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f2ffd700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 0x7fd8f419ad10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4000c00 tx=0x7fd8e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4027070 con 0x7fd8f41053d0 2024-04-25T10:53:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e40041d0 con 0x7fd8f41053d0 2024-04-25T10:53:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f9a1d700 1 -- 172.21.15.19:0/2913304064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f419fe70 con 0x7fd8f41053d0 2024-04-25T10:53:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400f460 con 0x7fd8f41053d0 2024-04-25T10:53:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.486+0000 7fd8f9a1d700 1 -- 172.21.15.19:0/2913304064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41a0350 con 0x7fd8f41053d0 2024-04-25T10:53:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.487+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8f4063e20 con 0x7fd8f41053d0 2024-04-25T10:53:07.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.489+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd8e402a070 con 0x7fd8f41053d0 2024-04-25T10:53:07.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f8a1b700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd8dc0afc10 0x7fd8dc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f27fc700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd8dc0afc10 0x7fd8dc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd8e40e0240 con 0x7fd8f41053d0 2024-04-25T10:53:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f8a1b700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] conn(0x7fd8dc0b9440 0x7fd8dc0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 --> [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8e40043d0 con 0x7fd8dc0b9440 2024-04-25T10:53:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd8e40e0720 con 0x7fd8f41053d0 2024-04-25T10:53:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f37fe700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] conn(0x7fd8dc0b9440 0x7fd8dc0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.490+0000 7fd8f27fc700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd8dc0afc10 0x7fd8dc0b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4102180 tx=0x7fd8e8005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.491+0000 7fd8f37fe700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] conn(0x7fd8dc0b9440 0x7fd8dc0bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.492+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== osd.7 v2:172.21.15.112:6816/2341909802 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8e40043d0 con 0x7fd8dc0b9440 2024-04-25T10:53:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:07 smithi169 ceph-mon[33049]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.508+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 --> [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd8f40642a0 con 0x7fd8dc0b9440 2024-04-25T10:53:07.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.508+0000 7fd8f8a1b700 1 -- 172.21.15.19:0/2913304064 <== osd.7 v2:172.21.15.112:6816/2341909802 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd8f40642a0 con 0x7fd8dc0b9440 2024-04-25T10:53:07.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] conn(0x7fd8dc0b9440 msgr2=0x7fd8dc0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] conn(0x7fd8dc0b9440 0x7fd8dc0bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd8dc0afc10 msgr2=0x7fd8dc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd8dc0afc10 0x7fd8dc0b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4102180 tx=0x7fd8e8005eb0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 msgr2=0x7fd8f419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 --2- 172.21.15.19:0/2913304064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8f41053d0 0x7fd8f419ad10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4000c00 tx=0x7fd8e40047b0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8f37fe700 1 -- 172.21.15.19:0/2913304064 reap_dead start 2024-04-25T10:53:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 shutdown_connections 2024-04-25T10:53:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 >> 172.21.15.19:0/2913304064 conn(0x7fd8f40fec90 msgr2=0x7fd8f40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 shutdown_connections 2024-04-25T10:53:07.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.509+0000 7fd8da7fc700 1 -- 172.21.15.19:0/2913304064 wait complete. 2024-04-25T10:53:07.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:07 smithi112 ceph-mon[32799]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.399+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/1104085313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 msgr2=0x7f6cdc10e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.399+0000 7f6ce0cfe700 1 --2- 172.21.15.19:0/1104085313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 0x7f6cdc10e220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009990 tx=0x7f6cc4009ca0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.400+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/1104085313 shutdown_connections 2024-04-25T10:53:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.400+0000 7f6ce0cfe700 1 --2- 172.21.15.19:0/1104085313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cdc10e7b0 0x7f6cdc110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.400+0000 7f6ce0cfe700 1 --2- 172.21.15.19:0/1104085313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc105380 0x7f6cdc105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.400+0000 7f6ce0cfe700 1 --2- 172.21.15.19:0/1104085313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 0x7f6cdc10e220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.400+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/1104085313 >> 172.21.15.19:0/1104085313 conn(0x7f6cdc0fec90 msgr2=0x7f6cdc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.400+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/1104085313 shutdown_connections 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.400+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/1104085313 wait complete. 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 Processor -- start 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 -- start start 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc105380 0x7f6cdc1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 0x7f6cdc1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cdc10e7b0 0x7f6cdc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc114350 con 0x7f6cdc105cd0 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc1141d0 con 0x7f6cdc105380 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6ce0cfe700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc1144d0 con 0x7f6cdc10e7b0 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 0x7f6cdc1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 0x7f6cdc1a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59150/0 (socket says 172.21.15.19:59150) 2024-04-25T10:53:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 -- 172.21.15.19:0/2322921468 learned_addr learned my addr 172.21.15.19:0/2322921468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cda59c700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc105380 0x7f6cdc1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cdad9d700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cdc10e7b0 0x7f6cdc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 -- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cdc10e7b0 msgr2=0x7f6cdc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cdc10e7b0 0x7f6cdc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 -- 172.21.15.19:0/2322921468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc105380 msgr2=0x7f6cdc1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc105380 0x7f6cdc1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.401+0000 7f6cd9d9b700 1 -- 172.21.15.19:0/2322921468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc19b280 con 0x7f6cdc105cd0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.402+0000 7f6cd9d9b700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 0x7f6cdc1a0f00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4005910 tx=0x7f6cc40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.402+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4027070 con 0x7f6cdc105cd0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.402+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc19b510 con 0x7f6cdc105cd0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.402+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc4003f20 con 0x7f6cdc105cd0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.402+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4022d00 con 0x7f6cdc105cd0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.402+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1a7770 con 0x7f6cdc105cd0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.403+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6cdc063e20 con 0x7f6cdc105cd0 2024-04-25T10:53:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.405+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6cc402a070 con 0x7f6cdc105cd0 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.405+0000 7f6cd37fe700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6cc80afc10 0x7f6cc80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.405+0000 7f6cda59c700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6cc80afc10 0x7f6cc80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.405+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f6cc40e08a0 con 0x7f6cdc105cd0 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.406+0000 7f6cd37fe700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] conn(0x7f6cc80b9440 0x7f6cc80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.406+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 --> [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cc40043d0 con 0x7f6cc80b9440 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.406+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f6cc40e0c50 con 0x7f6cdc105cd0 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.406+0000 7f6cda59c700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6cc80afc10 0x7f6cc80b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007be0 tx=0x7f6ccc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.406+0000 7f6cdad9d700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] conn(0x7f6cc80b9440 0x7f6cc80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.406+0000 7f6cdad9d700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] conn(0x7f6cc80b9440 0x7f6cc80bb890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.407+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== osd.11 v2:172.21.15.169:6824/2887715484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6cc40043d0 con 0x7f6cc80b9440 2024-04-25T10:53:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.424+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 --> [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6cdc064220 con 0x7f6cc80b9440 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.424+0000 7f6cd37fe700 1 -- 172.21.15.19:0/2322921468 <== osd.11 v2:172.21.15.169:6824/2887715484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6cdc064220 con 0x7f6cc80b9440 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] conn(0x7f6cc80b9440 msgr2=0x7f6cc80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] conn(0x7f6cc80b9440 0x7f6cc80bb890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6cc80afc10 msgr2=0x7f6cc80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6cc80afc10 0x7f6cc80b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007be0 tx=0x7f6ccc0073d0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 msgr2=0x7f6cdc1a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 --2- 172.21.15.19:0/2322921468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc105cd0 0x7f6cdc1a0f00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4005910 tx=0x7f6cc40047b0 comp rx=0 tx=0).stop 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6cdad9d700 1 -- 172.21.15.19:0/2322921468 reap_dead start 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 shutdown_connections 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 >> 172.21.15.19:0/2322921468 conn(0x7f6cdc0fec90 msgr2=0x7f6cdc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 shutdown_connections 2024-04-25T10:53:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:07.425+0000 7f6ce0cfe700 1 -- 172.21.15.19:0/2322921468 wait complete. 2024-04-25T10:53:08.275 INFO:teuthology.orchestra.run.smithi019.stdout:309237645332 2024-04-25T10:53:08.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T10:53:09.180 INFO:teuthology.orchestra.run.smithi019.stdout:206158430248 2024-04-25T10:53:09.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T10:53:09.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:09 smithi019 ceph-mon[29209]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:53:09.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:09 smithi169 ceph-mon[33049]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:53:09.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:09 smithi112 ceph-mon[32799]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:53:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 -- 172.21.15.19:0/65693610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c103360 msgr2=0x7fd04c103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 --2- 172.21.15.19:0/65693610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c103360 0x7fd04c103770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0440099c0 tx=0x7fd044009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 -- 172.21.15.19:0/65693610 shutdown_connections 2024-04-25T10:53:09.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 --2- 172.21.15.19:0/65693610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd04c1085e0 0x7fd04c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 --2- 172.21.15.19:0/65693610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c103360 0x7fd04c103770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 --2- 172.21.15.19:0/65693610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c107c90 0x7fd04c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 -- 172.21.15.19:0/65693610 >> 172.21.15.19:0/65693610 conn(0x7fd04c0fec90 msgr2=0x7fd04c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:09.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 -- 172.21.15.19:0/65693610 shutdown_connections 2024-04-25T10:53:09.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.789+0000 7fd053a11700 1 -- 172.21.15.19:0/65693610 wait complete. 2024-04-25T10:53:09.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 Processor -- start 2024-04-25T10:53:09.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 -- start start 2024-04-25T10:53:09.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c103360 0x7fd04c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:09.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd04c107c90 0x7fd04c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:09.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c1085e0 0x7fd04c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:09.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c114360 con 0x7fd04c1085e0 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1141e0 con 0x7fd04c107c90 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.790+0000 7fd053a11700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1144e0 con 0x7fd04c103360 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c1085e0 0x7fd04c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c1085e0 0x7fd04c19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59198/0 (socket says 172.21.15.19:59198) 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 -- 172.21.15.19:0/2230261225 learned_addr learned my addr 172.21.15.19:0/2230261225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd0517ad700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c103360 0x7fd04c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd050fac700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd04c107c90 0x7fd04c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 -- 172.21.15.19:0/2230261225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c103360 msgr2=0x7fd04c1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd04c103360 0x7fd04c1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 -- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd04c107c90 msgr2=0x7fd04c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd04c107c90 0x7fd04c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 -- 172.21.15.19:0/2230261225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c19b100 con 0x7fd04c1085e0 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd050fac700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd04c107c90 0x7fd04c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd051fae700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c1085e0 0x7fd04c19aac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007ee0 tx=0x7fd03c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00dbd0 con 0x7fd04c1085e0 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c19b3f0 con 0x7fd04c1085e0 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c00dd70 con 0x7fd04c1085e0 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.791+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c005990 con 0x7fd04c1085e0 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.792+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c19b8e0 con 0x7fd04c1085e0 2024-04-25T10:53:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.792+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd04c063e20 con 0x7fd04c1085e0 2024-04-25T10:53:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.794+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd03c01a020 con 0x7fd04c1085e0 2024-04-25T10:53:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.795+0000 7fd04a7fc700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd0340afc10 0x7fd0340b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.795+0000 7fd0517ad700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd0340afc10 0x7fd0340b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.795+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd03c0deb90 con 0x7fd04c1085e0 2024-04-25T10:53:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.795+0000 7fd04a7fc700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] conn(0x7fd0340b9440 0x7fd0340bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.795+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 --> [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd03c01ae50 con 0x7fd0340b9440 2024-04-25T10:53:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.795+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd03c0def40 con 0x7fd04c1085e0 2024-04-25T10:53:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.796+0000 7fd0517ad700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd0340afc10 0x7fd0340b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd04c19bc00 tx=0x7fd04401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.796+0000 7fd050fac700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] conn(0x7fd0340b9440 0x7fd0340bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.796+0000 7fd050fac700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] conn(0x7fd0340b9440 0x7fd0340bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.797+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== osd.10 v2:172.21.15.112:6824/2703835627 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd03c01ae50 con 0x7fd0340b9440 2024-04-25T10:53:09.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.820+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 --> [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd04c0642a0 con 0x7fd0340b9440 2024-04-25T10:53:09.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.821+0000 7fd04a7fc700 1 -- 172.21.15.19:0/2230261225 <== osd.10 v2:172.21.15.112:6824/2703835627 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd04c0642a0 con 0x7fd0340b9440 2024-04-25T10:53:09.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.821+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] conn(0x7fd0340b9440 msgr2=0x7fd0340bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.821+0000 7fd053a11700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] conn(0x7fd0340b9440 0x7fd0340bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.821+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd0340afc10 msgr2=0x7fd0340b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.821+0000 7fd053a11700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd0340afc10 0x7fd0340b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd04c19bc00 tx=0x7fd04401a040 comp rx=0 tx=0).stop 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.822+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c1085e0 msgr2=0x7fd04c19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.822+0000 7fd053a11700 1 --2- 172.21.15.19:0/2230261225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c1085e0 0x7fd04c19aac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007ee0 tx=0x7fd03c00d3b0 comp rx=0 tx=0).stop 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.822+0000 7fd051fae700 1 -- 172.21.15.19:0/2230261225 reap_dead start 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.822+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 shutdown_connections 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.822+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 >> 172.21.15.19:0/2230261225 conn(0x7fd04c0fec90 msgr2=0x7fd04c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.822+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 shutdown_connections 2024-04-25T10:53:09.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:09.822+0000 7fd053a11700 1 -- 172.21.15.19:0/2230261225 wait complete. 2024-04-25T10:53:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 -- 172.21.15.19:0/2893142072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 msgr2=0x7f1bdc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 --2- 172.21.15.19:0/2893142072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 0x7f1bdc1091d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0099c0 tx=0x7f1bcc009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 -- 172.21.15.19:0/2893142072 shutdown_connections 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 --2- 172.21.15.19:0/2893142072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bdc109710 0x7f1bdc111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 --2- 172.21.15.19:0/2893142072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc104490 0x7f1bdc1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 --2- 172.21.15.19:0/2893142072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 0x7f1bdc1091d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 -- 172.21.15.19:0/2893142072 >> 172.21.15.19:0/2893142072 conn(0x7f1bdc0ffd60 msgr2=0x7f1bdc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 -- 172.21.15.19:0/2893142072 shutdown_connections 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.984+0000 7f1be3cb0700 1 -- 172.21.15.19:0/2893142072 wait complete. 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 Processor -- start 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 -- start start 2024-04-25T10:53:10.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bdc104490 0x7f1bdc076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 0x7f1bdc076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc109710 0x7f1bdc077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc114320 con 0x7f1bdc108dc0 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1141a0 con 0x7f1bdc104490 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be3cb0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1144a0 con 0x7f1bdc109710 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 0x7f1bdc076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 0x7f1bdc076fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59216/0 (socket says 172.21.15.19:59216) 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 -- 172.21.15.19:0/1961753799 learned_addr learned my addr 172.21.15.19:0/1961753799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be1a4c700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bdc104490 0x7f1bdc076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be224d700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc109710 0x7f1bdc077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 -- 172.21.15.19:0/1961753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc109710 msgr2=0x7f1bdc077520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:10.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc109710 0x7f1bdc077520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 -- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bdc104490 msgr2=0x7f1bdc076aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bdc104490 0x7f1bdc076aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be124b700 1 -- 172.21.15.19:0/1961753799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc1a9030 con 0x7f1bdc108dc0 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.985+0000 7f1be224d700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc109710 0x7f1bdc077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.986+0000 7f1be124b700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 0x7f1bdc076fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006010 tx=0x7f1bcc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.986+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc027070 con 0x7f1bdc108dc0 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.986+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc1a92c0 con 0x7f1bdc108dc0 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.986+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc003780 con 0x7f1bdc108dc0 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.986+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc022be0 con 0x7f1bdc108dc0 2024-04-25T10:53:10.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.986+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc1a9770 con 0x7f1bdc108dc0 2024-04-25T10:53:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.987+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1bdc063d60 con 0x7f1bdc108dc0 2024-04-25T10:53:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.988+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1bcc02a070 con 0x7f1bdc108dc0 2024-04-25T10:53:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.989+0000 7f1bd2ffd700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1bc80afc10 0x7f1bc80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.989+0000 7f1be1a4c700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1bc80afc10 0x7f1bc80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.989+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f1bcc0e05e0 con 0x7f1bdc108dc0 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.989+0000 7f1bd2ffd700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] conn(0x7f1bc80b9440 0x7f1bc80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.989+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 --> [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bcc015ec0 con 0x7f1bc80b9440 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.989+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1bcc0e0ac0 con 0x7f1bdc108dc0 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.990+0000 7f1be1a4c700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1bc80afc10 0x7f1bc80b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005950 tx=0x7f1bd800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.990+0000 7f1be224d700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] conn(0x7f1bc80b9440 0x7f1bc80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.990+0000 7f1be224d700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] conn(0x7f1bc80b9440 0x7f1bc80bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:10.991+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== osd.1 v2:172.21.15.112:6800/3663453616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1bcc015ec0 con 0x7f1bc80b9440 2024-04-25T10:53:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.008+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 --> [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1bdc064210 con 0x7f1bc80b9440 2024-04-25T10:53:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.008+0000 7f1bd2ffd700 1 -- 172.21.15.19:0/1961753799 <== osd.1 v2:172.21.15.112:6800/3663453616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1bdc064210 con 0x7f1bc80b9440 2024-04-25T10:53:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.008+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] conn(0x7f1bc80b9440 msgr2=0x7f1bc80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.008+0000 7f1be3cb0700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] conn(0x7f1bc80b9440 0x7f1bc80bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1bc80afc10 msgr2=0x7f1bc80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1bc80afc10 0x7f1bc80b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005950 tx=0x7f1bd800b410 comp rx=0 tx=0).stop 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 msgr2=0x7f1bdc076fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 --2- 172.21.15.19:0/1961753799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc108dc0 0x7f1bdc076fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006010 tx=0x7f1bcc0047b0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be224d700 1 -- 172.21.15.19:0/1961753799 reap_dead start 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 shutdown_connections 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 >> 172.21.15.19:0/1961753799 conn(0x7f1bdc0ffd60 msgr2=0x7f1bdc10a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 shutdown_connections 2024-04-25T10:53:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.009+0000 7f1be3cb0700 1 -- 172.21.15.19:0/1961753799 wait complete. 2024-04-25T10:53:11.268 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:11 smithi019 ceph-mon[29209]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:11.361 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:11 smithi112 ceph-mon[32799]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:11.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:11 smithi169 ceph-mon[33049]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 -- 172.21.15.19:0/3801722585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 msgr2=0x7f9734102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 --2- 172.21.15.19:0/3801722585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 0x7f9734102e30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f972c00b0d0 tx=0x7f972c00b3e0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 -- 172.21.15.19:0/3801722585 shutdown_connections 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 --2- 172.21.15.19:0/3801722585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9734109720 0x7f9734111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 --2- 172.21.15.19:0/3801722585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 0x7f9734102e30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 --2- 172.21.15.19:0/3801722585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9734108dd0 0x7f97341091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 -- 172.21.15.19:0/3801722585 >> 172.21.15.19:0/3801722585 conn(0x7f97340fe5c0 msgr2=0x7f97341009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 -- 172.21.15.19:0/3801722585 shutdown_connections 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.888+0000 7f973b9c5700 1 -- 172.21.15.19:0/3801722585 wait complete. 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 Processor -- start 2024-04-25T10:53:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 -- start start 2024-04-25T10:53:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 0x7f9734103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9734108dd0 0x7f9734103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9734109720 0x7f97341043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97341143d0 con 0x7f9734102a20 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9734114250 con 0x7f9734109720 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f973b9c5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9734114550 con 0x7f9734108dd0 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.889+0000 7f9739761700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 0x7f9734103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 0x7f9734103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59228/0 (socket says 172.21.15.19:59228) 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 -- 172.21.15.19:0/861097483 learned_addr learned my addr 172.21.15.19:0/861097483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9738f60700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9734108dd0 0x7f9734103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739f62700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9734109720 0x7f97341043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 -- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9734108dd0 msgr2=0x7f9734103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9734108dd0 0x7f9734103eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 -- 172.21.15.19:0/861097483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9734109720 msgr2=0x7f97341043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9734109720 0x7f97341043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 -- 172.21.15.19:0/861097483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97341a90f0 con 0x7f9734102a20 2024-04-25T10:53:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f9739761700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 0x7f9734103970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f972c007a20 tx=0x7f972c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c01f070 con 0x7f9734102a20 2024-04-25T10:53:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f972c003c00 con 0x7f9734102a20 2024-04-25T10:53:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97341a9380 con 0x7f9734102a20 2024-04-25T10:53:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c01b730 con 0x7f9734102a20 2024-04-25T10:53:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.890+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97341a97d0 con 0x7f9734102a20 2024-04-25T10:53:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.891+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9734063da0 con 0x7f9734102a20 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.893+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f972c022070 con 0x7f9734102a20 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.893+0000 7f97327fc700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f971c0afc10 0x7f971c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.893+0000 7f9738f60700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f971c0afc10 0x7f971c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.893+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f972c00f020 con 0x7f9734102a20 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.894+0000 7f97327fc700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] conn(0x7f971c0b9440 0x7f971c0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.894+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 --> [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f972c02ce50 con 0x7f971c0b9440 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.894+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f972c0e0660 con 0x7f9734102a20 2024-04-25T10:53:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.894+0000 7f9738f60700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f971c0afc10 0x7f971c0b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9728005ea0 tx=0x7f9728005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.894+0000 7f9739f62700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] conn(0x7f971c0b9440 0x7f971c0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.895+0000 7f9739f62700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] conn(0x7f971c0b9440 0x7f971c0bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.895+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== osd.8 v2:172.21.15.169:6816/4055159566 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f972c02ce50 con 0x7f971c0b9440 2024-04-25T10:53:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.920+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 --> [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97340641a0 con 0x7f971c0b9440 2024-04-25T10:53:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f97327fc700 1 -- 172.21.15.19:0/861097483 <== osd.8 v2:172.21.15.169:6816/4055159566 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f97340641a0 con 0x7f971c0b9440 2024-04-25T10:53:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] conn(0x7f971c0b9440 msgr2=0x7f971c0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] conn(0x7f971c0b9440 0x7f971c0bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f971c0afc10 msgr2=0x7f971c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f971c0afc10 0x7f971c0b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9728005ea0 tx=0x7f9728005e30 comp rx=0 tx=0).stop 2024-04-25T10:53:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 msgr2=0x7f9734103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 --2- 172.21.15.19:0/861097483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734102a20 0x7f9734103970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f972c007a20 tx=0x7f972c007b00 comp rx=0 tx=0).stop 2024-04-25T10:53:11.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f9739f62700 1 -- 172.21.15.19:0/861097483 reap_dead start 2024-04-25T10:53:11.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 shutdown_connections 2024-04-25T10:53:11.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 >> 172.21.15.19:0/861097483 conn(0x7f97340fe5c0 msgr2=0x7f9734108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:11.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 shutdown_connections 2024-04-25T10:53:11.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:11.921+0000 7f973b9c5700 1 -- 172.21.15.19:0/861097483 wait complete. 2024-04-25T10:53:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:283467841561 2024-04-25T10:53:12.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 -- 172.21.15.19:0/339056333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9270103360 msgr2=0x7f9270103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 --2- 172.21.15.19:0/339056333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9270103360 0x7f9270103770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92600099c0 tx=0x7f9260009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 -- 172.21.15.19:0/339056333 shutdown_connections 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 --2- 172.21.15.19:0/339056333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92701085e0 0x7f927010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 --2- 172.21.15.19:0/339056333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9270103360 0x7f9270103770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 --2- 172.21.15.19:0/339056333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270107c90 0x7f92701080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 -- 172.21.15.19:0/339056333 >> 172.21.15.19:0/339056333 conn(0x7f92700fecb0 msgr2=0x7f92701010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 -- 172.21.15.19:0/339056333 shutdown_connections 2024-04-25T10:53:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.123+0000 7f9275a50700 1 -- 172.21.15.19:0/339056333 wait complete. 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 Processor -- start 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 -- start start 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270103360 0x7f927019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270107c90 0x7f927019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92701085e0 0x7f927019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9270113210 con 0x7f92701085e0 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9270113090 con 0x7f9270103360 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f9275a50700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9270113390 con 0x7f9270107c90 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92701085e0 0x7f927019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92701085e0 0x7f927019c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59240/0 (socket says 172.21.15.19:59240) 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 -- 172.21.15.19:0/1142113704 learned_addr learned my addr 172.21.15.19:0/1142113704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926effd700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270103360 0x7f927019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926e7fc700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270107c90 0x7f927019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 -- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270107c90 msgr2=0x7f927019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270107c90 0x7f927019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 -- 172.21.15.19:0/1142113704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270103360 msgr2=0x7f927019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9270103360 0x7f927019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926f7fe700 1 -- 172.21.15.19:0/1142113704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92701a6e60 con 0x7f92701085e0 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.124+0000 7f926e7fc700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9270107c90 0x7f927019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.125+0000 7f926f7fe700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92701085e0 0x7f927019c5b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f926800ef30 tx=0x7f926800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.125+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92680092e0 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.125+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92701a70f0 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.125+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9268004bb0 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.125+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268004280 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.125+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92701a7630 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.126+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9270063da0 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.127+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f926801b030 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.128+0000 7f9274a4e700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92580afc10 0x7f92580b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.128+0000 7f926effd700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92580afc10 0x7f92580b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.128+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f92680df5c0 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.128+0000 7f9274a4e700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] conn(0x7f92580b9440 0x7f92580bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.128+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 --> [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9268008ec0 con 0x7f92580b9440 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.128+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f92680dfaa0 con 0x7f92701085e0 2024-04-25T10:53:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.129+0000 7f926effd700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92580afc10 0x7f92580b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9260006010 tx=0x7f926001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.129+0000 7f926e7fc700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] conn(0x7f92580b9440 0x7f92580bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.129+0000 7f926e7fc700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] conn(0x7f92580b9440 0x7f92580bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.130+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== osd.2 v2:172.21.15.169:6800/3608720428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9268008ec0 con 0x7f92580b9440 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.146+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 --> [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92700641a0 con 0x7f92580b9440 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9274a4e700 1 -- 172.21.15.19:0/1142113704 <== osd.2 v2:172.21.15.169:6800/3608720428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f92700641a0 con 0x7f92580b9440 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] conn(0x7f92580b9440 msgr2=0x7f92580bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] conn(0x7f92580b9440 0x7f92580bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92580afc10 msgr2=0x7f92580b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f92580afc10 0x7f92580b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9260006010 tx=0x7f926001a040 comp rx=0 tx=0).stop 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92701085e0 msgr2=0x7f927019c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 --2- 172.21.15.19:0/1142113704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92701085e0 0x7f927019c5b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f926800ef30 tx=0x7f926800c450 comp rx=0 tx=0).stop 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f926f7fe700 1 -- 172.21.15.19:0/1142113704 reap_dead start 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 shutdown_connections 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.147+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 >> 172.21.15.19:0/1142113704 conn(0x7f92700fecb0 msgr2=0x7f9270109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.148+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 shutdown_connections 2024-04-25T10:53:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:12.148+0000 7f9275a50700 1 -- 172.21.15.19:0/1142113704 wait complete. 2024-04-25T10:53:13.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:13 smithi019 ceph-mon[29209]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:13.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:13 smithi169 ceph-mon[33049]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 -- 172.21.15.19:0/1406356607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c104480 msgr2=0x7fe62c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 --2- 172.21.15.19:0/1406356607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c104480 0x7fe62c104890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0099c0 tx=0x7fe61c009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 -- 172.21.15.19:0/1406356607 shutdown_connections 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 --2- 172.21.15.19:0/1406356607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c109700 0x7fe62c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 --2- 172.21.15.19:0/1406356607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c104480 0x7fe62c104890 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 --2- 172.21.15.19:0/1406356607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe62c108db0 0x7fe62c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 -- 172.21.15.19:0/1406356607 >> 172.21.15.19:0/1406356607 conn(0x7fe62c0ffd70 msgr2=0x7fe62c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 -- 172.21.15.19:0/1406356607 shutdown_connections 2024-04-25T10:53:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 -- 172.21.15.19:0/1406356607 wait complete. 2024-04-25T10:53:13.557 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:13 smithi112 ceph-mon[32799]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:53:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.555+0000 7fe63359b700 1 Processor -- start 2024-04-25T10:53:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe63359b700 1 -- start start 2024-04-25T10:53:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe63359b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c104480 0x7fe62c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe63359b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe62c108db0 0x7fe62c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe63359b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c109700 0x7fe62c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe63359b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c114330 con 0x7fe62c109700 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe63359b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1141b0 con 0x7fe62c104480 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe63359b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1144b0 con 0x7fe62c108db0 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c109700 0x7fe62c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c109700 0x7fe62c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51716/0 (socket says 172.21.15.19:51716) 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 -- 172.21.15.19:0/3954999547 learned_addr learned my addr 172.21.15.19:0/3954999547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631337700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c104480 0x7fe62c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe630b36700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe62c108db0 0x7fe62c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 -- 172.21.15.19:0/3954999547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe62c108db0 msgr2=0x7fe62c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe62c108db0 0x7fe62c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 -- 172.21.15.19:0/3954999547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c104480 msgr2=0x7fe62c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c104480 0x7fe62c1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 -- 172.21.15.19:0/3954999547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c19b0f0 con 0x7fe62c109700 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.556+0000 7fe631b38700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c109700 0x7fe62c19aab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe624007ee0 tx=0x7fe62400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.557+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624004bb0 con 0x7fe62c109700 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.557+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe624004d50 con 0x7fe62c109700 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.557+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624005920 con 0x7fe62c109700 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.557+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c19b3e0 con 0x7fe62c109700 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.557+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c19b8d0 con 0x7fe62c109700 2024-04-25T10:53:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.557+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe62c063e20 con 0x7fe62c109700 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.559+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe62401a020 con 0x7fe62c109700 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe6227fc700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe6180afc90 0x7fe6180b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe631337700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe6180afc90 0x7fe6180b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fe6240de880 con 0x7fe62c109700 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe6227fc700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] conn(0x7fe6180b94c0 0x7fe6180bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 --> [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe624021e70 con 0x7fe6180b94c0 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe6240ded60 con 0x7fe62c109700 2024-04-25T10:53:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe630b36700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] conn(0x7fe6180b94c0 0x7fe6180bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.560+0000 7fe631337700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe6180afc90 0x7fe6180b21a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe62c19bbf0 tx=0x7fe61c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.561+0000 7fe630b36700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] conn(0x7fe6180b94c0 0x7fe6180bb910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.562+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== osd.6 v2:172.21.15.19:6818/2613393193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe624021e70 con 0x7fe6180b94c0 2024-04-25T10:53:13.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.578+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 --> [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe62c0642a0 con 0x7fe6180b94c0 2024-04-25T10:53:13.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.578+0000 7fe6227fc700 1 -- 172.21.15.19:0/3954999547 <== osd.6 v2:172.21.15.19:6818/2613393193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe62c0642a0 con 0x7fe6180b94c0 2024-04-25T10:53:13.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.578+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] conn(0x7fe6180b94c0 msgr2=0x7fe6180bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:13.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.578+0000 7fe63359b700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] conn(0x7fe6180b94c0 0x7fe6180bb910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe6180afc90 msgr2=0x7fe6180b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe6180afc90 0x7fe6180b21a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe62c19bbf0 tx=0x7fe61c01a040 comp rx=0 tx=0).stop 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c109700 msgr2=0x7fe62c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 --2- 172.21.15.19:0/3954999547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe62c109700 0x7fe62c19aab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe624007ee0 tx=0x7fe62400d3b0 comp rx=0 tx=0).stop 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe631b38700 1 -- 172.21.15.19:0/3954999547 reap_dead start 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 shutdown_connections 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 >> 172.21.15.19:0/3954999547 conn(0x7fe62c0ffd70 msgr2=0x7fe62c108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 shutdown_connections 2024-04-25T10:53:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:13.579+0000 7fe63359b700 1 -- 172.21.15.19:0/3954999547 wait complete. 2024-04-25T10:53:14.750 INFO:teuthology.orchestra.run.smithi019.stdout:55834574918 2024-04-25T10:53:14.750 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T10:53:14.923 INFO:teuthology.orchestra.run.smithi019.stdout:231928234020 2024-04-25T10:53:14.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T10:53:15.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:15 smithi019 ceph-mon[29209]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:15.468 INFO:teuthology.orchestra.run.smithi019.stdout:77309411395 2024-04-25T10:53:15.469 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T10:53:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:15 smithi169 ceph-mon[33049]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:15.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:15 smithi112 ceph-mon[32799]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:15.953 INFO:teuthology.orchestra.run.smithi019.stdout:184683593775 2024-04-25T10:53:15.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T10:53:16.366 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:16.369 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:17.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:17 smithi019 ceph-mon[29209]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:17 smithi169 ceph-mon[33049]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:17 smithi112 ceph-mon[32799]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 -- 172.21.15.19:0/2578549003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec103360 msgr2=0x7f4fec103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 --2- 172.21.15.19:0/2578549003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec103360 0x7f4fec103770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40099c0 tx=0x7f4fd4009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 -- 172.21.15.19:0/2578549003 shutdown_connections 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 --2- 172.21.15.19:0/2578549003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fec1085e0 0x7f4fec10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 --2- 172.21.15.19:0/2578549003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec103360 0x7f4fec103770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 --2- 172.21.15.19:0/2578549003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fec107c90 0x7f4fec1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 -- 172.21.15.19:0/2578549003 >> 172.21.15.19:0/2578549003 conn(0x7f4fec0fec90 msgr2=0x7f4fec1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 -- 172.21.15.19:0/2578549003 shutdown_connections 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.634+0000 7f4ff1524700 1 -- 172.21.15.19:0/2578549003 wait complete. 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 Processor -- start 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 -- start start 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fec103360 0x7f4fec1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fec107c90 0x7f4fec1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec1085e0 0x7f4fec19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec114360 con 0x7f4fec1085e0 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1141e0 con 0x7f4fec107c90 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.635+0000 7f4ff1524700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1144e0 con 0x7f4fec103360 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec1085e0 0x7f4fec19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec1085e0 0x7f4fec19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51748/0 (socket says 172.21.15.19:51748) 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 -- 172.21.15.19:0/1865529103 learned_addr learned my addr 172.21.15.19:0/1865529103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4fea7fc700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fec107c90 0x7f4fec1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feaffd700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fec103360 0x7f4fec1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 -- 172.21.15.19:0/1865529103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fec103360 msgr2=0x7f4fec1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fec103360 0x7f4fec1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 -- 172.21.15.19:0/1865529103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fec107c90 msgr2=0x7f4fec1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fec107c90 0x7f4fec1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 -- 172.21.15.19:0/1865529103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec19b150 con 0x7f4fec1085e0 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4feb7fe700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec1085e0 0x7f4fec19aae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4007d70 tx=0x7f4fe400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4016020 con 0x7f4fec1085e0 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec19b440 con 0x7f4fec1085e0 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe400ac90 con 0x7f4fec1085e0 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40115e0 con 0x7f4fec1085e0 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.636+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a7750 con 0x7f4fec1085e0 2024-04-25T10:53:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.637+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4fec063e20 con 0x7f4fec1085e0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.639+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4fe4019020 con 0x7f4fec1085e0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.639+0000 7f4fe3fff700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4fd80afc10 0x7f4fd80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4feaffd700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4fd80afc10 0x7f4fd80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f4fe40defc0 con 0x7f4fec1085e0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4fe3fff700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] conn(0x7f4fd80b9440 0x7f4fd80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 --> [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4fe4017ec0 con 0x7f4fd80b9440 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4fe40df4a0 con 0x7f4fec1085e0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4fea7fc700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] conn(0x7f4fd80b9440 0x7f4fd80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4feaffd700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4fd80afc10 0x7f4fd80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4fec19bc20 tx=0x7f4fd401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.640+0000 7f4fea7fc700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] conn(0x7f4fd80b9440 0x7f4fd80bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.641+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== osd.0 v2:172.21.15.19:6802/2420046240 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4fe4017ec0 con 0x7f4fd80b9440 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.657+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 --> [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4fec0642a0 con 0x7f4fd80b9440 2024-04-25T10:53:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4fe3fff700 1 -- 172.21.15.19:0/1865529103 <== osd.0 v2:172.21.15.19:6802/2420046240 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4fec0642a0 con 0x7f4fd80b9440 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] conn(0x7f4fd80b9440 msgr2=0x7f4fd80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] conn(0x7f4fd80b9440 0x7f4fd80bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4fd80afc10 msgr2=0x7f4fd80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4fd80afc10 0x7f4fd80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4fec19bc20 tx=0x7f4fd401a040 comp rx=0 tx=0).stop 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec1085e0 msgr2=0x7f4fec19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 --2- 172.21.15.19:0/1865529103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fec1085e0 0x7f4fec19aae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4007d70 tx=0x7f4fe400a340 comp rx=0 tx=0).stop 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4feb7fe700 1 -- 172.21.15.19:0/1865529103 reap_dead start 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 shutdown_connections 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 >> 172.21.15.19:0/1865529103 conn(0x7f4fec0fec90 msgr2=0x7f4fec1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 shutdown_connections 2024-04-25T10:53:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:17.658+0000 7f4ff1524700 1 -- 172.21.15.19:0/1865529103 wait complete. 2024-04-25T10:53:19.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:19 smithi019 ceph-mon[29209]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:19.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:19 smithi169 ceph-mon[33049]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:19.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:19 smithi112 ceph-mon[32799]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:21 smithi019 ceph-mon[29209]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:21 smithi169 ceph-mon[33049]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:21 smithi112 ceph-mon[32799]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:22.387 INFO:teuthology.orchestra.run.smithi019.stdout:34359738446 2024-04-25T10:53:22.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T10:53:23.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:23 smithi019 ceph-mon[29209]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:23 smithi169 ceph-mon[33049]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:23 smithi112 ceph-mon[32799]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 -- 172.21.15.19:0/4280465285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 msgr2=0x7f944c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 --2- 172.21.15.19:0/4280465285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c105710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f944400b0a0 tx=0x7f944400b3b0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 -- 172.21.15.19:0/4280465285 shutdown_connections 2024-04-25T10:53:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 --2- 172.21.15.19:0/4280465285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c10e770 0x7f944c110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 --2- 172.21.15.19:0/4280465285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c105710 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 --2- 172.21.15.19:0/4280465285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944c105c50 0x7f944c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 -- 172.21.15.19:0/4280465285 >> 172.21.15.19:0/4280465285 conn(0x7f944c0fec90 msgr2=0x7f944c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 -- 172.21.15.19:0/4280465285 shutdown_connections 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.279+0000 7f94533ad700 1 -- 172.21.15.19:0/4280465285 wait complete. 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 Processor -- start 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 -- start start 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944c105c50 0x7f944c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c10e770 0x7f944c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f944c114340 con 0x7f944c105300 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1141c0 con 0x7f944c105c50 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.280+0000 7f94533ad700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1144c0 con 0x7f944c10e770 2024-04-25T10:53:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c1a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45964/0 (socket says 172.21.15.19:45964) 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 -- 172.21.15.19:0/4292332171 learned_addr learned my addr 172.21.15.19:0/4292332171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9450948700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944c105c50 0x7f944c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f945194a700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c10e770 0x7f944c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 -- 172.21.15.19:0/4292332171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c10e770 msgr2=0x7f944c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c10e770 0x7f944c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 -- 172.21.15.19:0/4292332171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944c105c50 msgr2=0x7f944c1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944c105c50 0x7f944c1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 -- 172.21.15.19:0/4292332171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c19b1c0 con 0x7f944c105300 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f945194a700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c10e770 0x7f944c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f9451149700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c1a08a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9444006070 tx=0x7f9444004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f944a7fc700 1 -- 172.21.15.19:0/4292332171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444021070 con 0x7f944c105300 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c19b450 con 0x7f944c105300 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f944a7fc700 1 -- 172.21.15.19:0/4292332171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9444007df0 con 0x7f944c105300 2024-04-25T10:53:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f944a7fc700 1 -- 172.21.15.19:0/4292332171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944401d450 con 0x7f944c105300 2024-04-25T10:53:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.281+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1a7750 con 0x7f944c105300 2024-04-25T10:53:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.282+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c051190 con 0x7f944c105300 2024-04-25T10:53:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.284+0000 7f944a7fc700 1 -- 172.21.15.19:0/4292332171 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f94440040b0 con 0x7f944c105300 2024-04-25T10:53:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.284+0000 7f944a7fc700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f94340afc10 0x7f94340b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.285+0000 7f9450948700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f94340afc10 0x7f94340b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.285+0000 7f944a7fc700 1 -- 172.21.15.19:0/4292332171 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f944400f020 con 0x7f944c105300 2024-04-25T10:53:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.285+0000 7f9450948700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f94340afc10 0x7f94340b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9440006fd0 tx=0x7f9440008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:24.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.288+0000 7f944a7fc700 1 -- 172.21.15.19:0/4292332171 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f944402a020 con 0x7f944c105300 2024-04-25T10:53:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.498+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f944c02e510 con 0x7f944c105300 2024-04-25T10:53:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.499+0000 7f944a7fc700 1 -- 172.21.15.19:0/4292332171 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94440a9440 con 0x7f944c105300 2024-04-25T10:53:24.500 INFO:teuthology.orchestra.run.smithi019.stdout:158913790006 2024-04-25T10:53:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f94340afc10 msgr2=0x7f94340b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f94340afc10 0x7f94340b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9440006fd0 tx=0x7f9440008040 comp rx=0 tx=0).stop 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 msgr2=0x7f944c1a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c1a08a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9444006070 tx=0x7f9444004ef0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 shutdown_connections 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f94340afc10 0x7f94340b2120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c10e770 0x7f944c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c105300 0x7f944c1a08a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 --2- 172.21.15.19:0/4292332171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944c105c50 0x7f944c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 >> 172.21.15.19:0/4292332171 conn(0x7f944c0fec90 msgr2=0x7f944c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 shutdown_connections 2024-04-25T10:53:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.501+0000 7f94533ad700 1 -- 172.21.15.19:0/4292332171 wait complete. 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 -- 172.21.15.19:0/1637933329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4103390 msgr2=0x7fbcc41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 --2- 172.21.15.19:0/1637933329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4103390 0x7fbcc41037a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00099c0 tx=0x7fbcc0009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 -- 172.21.15.19:0/1637933329 shutdown_connections 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 --2- 172.21.15.19:0/1637933329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc4108610 0x7fbcc410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 --2- 172.21.15.19:0/1637933329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4103390 0x7fbcc41037a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 --2- 172.21.15.19:0/1637933329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcc4107cc0 0x7fbcc41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 -- 172.21.15.19:0/1637933329 >> 172.21.15.19:0/1637933329 conn(0x7fbcc40fec90 msgr2=0x7fbcc41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 -- 172.21.15.19:0/1637933329 shutdown_connections 2024-04-25T10:53:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 -- 172.21.15.19:0/1637933329 wait complete. 2024-04-25T10:53:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.582+0000 7fbccbf2c700 1 Processor -- start 2024-04-25T10:53:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbccbf2c700 1 -- start start 2024-04-25T10:53:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbccbf2c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcc4103390 0x7fbcc41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbccbf2c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107cc0 0x7fbcc4199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbccbf2c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc4108610 0x7fbcc419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbccbf2c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4113240 con 0x7fbcc4107cc0 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbccbf2c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc41130c0 con 0x7fbcc4108610 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbccbf2c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc41133c0 con 0x7fbcc4103390 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107cc0 0x7fbcc4199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107cc0 0x7fbcc4199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45982/0 (socket says 172.21.15.19:45982) 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 -- 172.21.15.19:0/2608234540 learned_addr learned my addr 172.21.15.19:0/2608234540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcca4c9700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc4108610 0x7fbcc419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc9cc8700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcc4103390 0x7fbcc41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 -- 172.21.15.19:0/2608234540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcc4103390 msgr2=0x7fbcc41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcc4103390 0x7fbcc41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 -- 172.21.15.19:0/2608234540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc4108610 msgr2=0x7fbcc419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc4108610 0x7fbcc419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.583+0000 7fbcc94c7700 1 -- 172.21.15.19:0/2608234540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc41a8ff0 con 0x7fbcc4107cc0 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.584+0000 7fbcc94c7700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107cc0 0x7fbcc4199e30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c8f0 tx=0x7fbcb400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.584+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40078d0 con 0x7fbcc4107cc0 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.584+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc41a92e0 con 0x7fbcc4107cc0 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.584+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb400f450 con 0x7fbcc4107cc0 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.584+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4018660 con 0x7fbcc4107cc0 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.584+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc41a9870 con 0x7fbcc4107cc0 2024-04-25T10:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.585+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbcc4063e20 con 0x7fbcc4107cc0 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.586+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fbcb400f900 con 0x7fbcc4107cc0 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.587+0000 7fbcbaffd700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbcb00afc10 0x7fbcb00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.587+0000 7fbcc9cc8700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbcb00afc10 0x7fbcb00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.587+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fbcb40deca0 con 0x7fbcc4107cc0 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.587+0000 7fbcbaffd700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] conn(0x7fbcb00b9440 0x7fbcb00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.587+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 --> [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbcb4015e70 con 0x7fbcb00b9440 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.587+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fbcb40df180 con 0x7fbcc4107cc0 2024-04-25T10:53:24.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.588+0000 7fbcca4c9700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] conn(0x7fbcb00b9440 0x7fbcb00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:24.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.588+0000 7fbcc9cc8700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbcb00afc10 0x7fbcb00b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009990 tx=0x7fbcc00036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:24.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.588+0000 7fbcca4c9700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] conn(0x7fbcb00b9440 0x7fbcb00bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:24.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.589+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== osd.3 v2:172.21.15.19:6810/930248439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcb4015e70 con 0x7fbcb00b9440 2024-04-25T10:53:24.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.605+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 --> [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbcc4064220 con 0x7fbcb00b9440 2024-04-25T10:53:24.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.605+0000 7fbcbaffd700 1 -- 172.21.15.19:0/2608234540 <== osd.3 v2:172.21.15.19:6810/930248439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbcc4064220 con 0x7fbcb00b9440 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.605+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] conn(0x7fbcb00b9440 msgr2=0x7fbcb00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.605+0000 7fbccbf2c700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] conn(0x7fbcb00b9440 0x7fbcb00bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbcb00afc10 msgr2=0x7fbcb00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbcb00afc10 0x7fbcb00b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009990 tx=0x7fbcc00036e0 comp rx=0 tx=0).stop 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107cc0 msgr2=0x7fbcc4199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 --2- 172.21.15.19:0/2608234540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107cc0 0x7fbcc4199e30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c8f0 tx=0x7fbcb400cc00 comp rx=0 tx=0).stop 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbcca4c9700 1 -- 172.21.15.19:0/2608234540 reap_dead start 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 shutdown_connections 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 >> 172.21.15.19:0/2608234540 conn(0x7fbcc40fec90 msgr2=0x7fbcc4101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 shutdown_connections 2024-04-25T10:53:24.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:24.606+0000 7fbccbf2c700 1 -- 172.21.15.19:0/2608234540 wait complete. 2024-04-25T10:53:25.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:25 smithi019 ceph-mon[29209]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:25.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:25 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4292332171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T10:53:25.427 INFO:teuthology.orchestra.run.smithi019.stdout:111669149759 2024-04-25T10:53:25.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T10:53:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:25 smithi169 ceph-mon[33049]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:25 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4292332171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T10:53:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:25 smithi112 ceph-mon[32799]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:25 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4292332171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T10:53:26.871 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790001 got 158913790006 for osd.5 2024-04-25T10:53:26.871 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:27.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:27 smithi019 ceph-mon[29209]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:27 smithi169 ceph-mon[33049]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:27.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:27 smithi112 ceph-mon[32799]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 -- 172.21.15.19:0/2908213705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 msgr2=0x7fea6410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 --2- 172.21.15.19:0/2908213705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6410ff70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b0d0 tx=0x7fea5c00b3e0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 -- 172.21.15.19:0/2908213705 shutdown_connections 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 --2- 172.21.15.19:0/2908213705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6410ff70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 --2- 172.21.15.19:0/2908213705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea64103390 0x7fea641037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 --2- 172.21.15.19:0/2908213705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea64107cc0 0x7fea641080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 -- 172.21.15.19:0/2908213705 >> 172.21.15.19:0/2908213705 conn(0x7fea640fec90 msgr2=0x7fea641010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:27.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 -- 172.21.15.19:0/2908213705 shutdown_connections 2024-04-25T10:53:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.604+0000 7fea68c98700 1 -- 172.21.15.19:0/2908213705 wait complete. 2024-04-25T10:53:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.605+0000 7fea68c98700 1 Processor -- start 2024-04-25T10:53:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.605+0000 7fea68c98700 1 -- start start 2024-04-25T10:53:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea68c98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea64103390 0x7fea641a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea68c98700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea64107cc0 0x7fea641a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea68c98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea68c98700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea64114390 con 0x7fea64108610 2024-04-25T10:53:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea68c98700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea64114210 con 0x7fea64103390 2024-04-25T10:53:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea68c98700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea64114510 con 0x7fea64107cc0 2024-04-25T10:53:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46006/0 (socket says 172.21.15.19:46006) 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 -- 172.21.15.19:0/332691101 learned_addr learned my addr 172.21.15.19:0/332691101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea61d9b700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea64107cc0 0x7fea641a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea6259c700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea64103390 0x7fea641a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 -- 172.21.15.19:0/332691101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea64107cc0 msgr2=0x7fea641a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea64107cc0 0x7fea641a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 -- 172.21.15.19:0/332691101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea64103390 msgr2=0x7fea641a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea64103390 0x7fea641a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 -- 172.21.15.19:0/332691101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6419b140 con 0x7fea64108610 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.606+0000 7fea62d9d700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6419aad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009fd0 tx=0x7fea5c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.607+0000 7fea537fe700 1 -- 172.21.15.19:0/332691101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c01f070 con 0x7fea64108610 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.607+0000 7fea537fe700 1 -- 172.21.15.19:0/332691101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c007e50 con 0x7fea64108610 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.607+0000 7fea537fe700 1 -- 172.21.15.19:0/332691101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c01b6a0 con 0x7fea64108610 2024-04-25T10:53:27.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.607+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6419b3d0 con 0x7fea64108610 2024-04-25T10:53:27.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.607+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6419b8b0 con 0x7fea64108610 2024-04-25T10:53:27.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.608+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea64051190 con 0x7fea64108610 2024-04-25T10:53:27.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.609+0000 7fea537fe700 1 -- 172.21.15.19:0/332691101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fea5c022070 con 0x7fea64108610 2024-04-25T10:53:27.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.610+0000 7fea537fe700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fea4c0afc10 0x7fea4c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:27.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.610+0000 7fea6259c700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fea4c0afc10 0x7fea4c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:27.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.610+0000 7fea537fe700 1 -- 172.21.15.19:0/332691101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fea5c00f020 con 0x7fea64108610 2024-04-25T10:53:27.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.610+0000 7fea6259c700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fea4c0afc10 0x7fea4c0b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea6419bc10 tx=0x7fea5400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.615+0000 7fea537fe700 1 -- 172.21.15.19:0/332691101 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea5c029020 con 0x7fea64108610 2024-04-25T10:53:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.829+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fea6402e510 con 0x7fea64108610 2024-04-25T10:53:27.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.829+0000 7fea537fe700 1 -- 172.21.15.19:0/332691101 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fea5c0a8b40 con 0x7fea64108610 2024-04-25T10:53:27.831 INFO:teuthology.orchestra.run.smithi019.stdout:257698037795 2024-04-25T10:53:27.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fea4c0afc10 msgr2=0x7fea4c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:27.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fea4c0afc10 0x7fea4c0b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea6419bc10 tx=0x7fea5400a380 comp rx=0 tx=0).stop 2024-04-25T10:53:27.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 msgr2=0x7fea6419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:27.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6419aad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009fd0 tx=0x7fea5c007a20 comp rx=0 tx=0).stop 2024-04-25T10:53:27.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 shutdown_connections 2024-04-25T10:53:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fea4c0afc10 0x7fea4c0b2120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea64108610 0x7fea6419aad0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea64103390 0x7fea641a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 --2- 172.21.15.19:0/332691101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea64107cc0 0x7fea641a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 >> 172.21.15.19:0/332691101 conn(0x7fea640fec90 msgr2=0x7fea64107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.831+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 shutdown_connections 2024-04-25T10:53:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:27.832+0000 7fea68c98700 1 -- 172.21.15.19:0/332691101 wait complete. 2024-04-25T10:53:28.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:28 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/332691101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T10:53:28.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:28 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/332691101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T10:53:28.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:28 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/332691101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 -- 172.21.15.19:0/2252313429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4103390 msgr2=0x7fcad41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 --2- 172.21.15.19:0/2252313429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4103390 0x7fcad41037a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcac40099c0 tx=0x7fcac4009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 -- 172.21.15.19:0/2252313429 shutdown_connections 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 --2- 172.21.15.19:0/2252313429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad4108610 0x7fcad410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 --2- 172.21.15.19:0/2252313429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4103390 0x7fcad41037a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 --2- 172.21.15.19:0/2252313429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcad4107cc0 0x7fcad41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 -- 172.21.15.19:0/2252313429 >> 172.21.15.19:0/2252313429 conn(0x7fcad40fecc0 msgr2=0x7fcad41010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 -- 172.21.15.19:0/2252313429 shutdown_connections 2024-04-25T10:53:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.618+0000 7fcad9182700 1 -- 172.21.15.19:0/2252313429 wait complete. 2024-04-25T10:53:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 Processor -- start 2024-04-25T10:53:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 -- start start 2024-04-25T10:53:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad4103390 0x7fcad40ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4107cc0 0x7fcad40ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcad4108610 0x7fcad40ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41132c0 con 0x7fcad4107cc0 2024-04-25T10:53:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113140 con 0x7fcad4103390 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad9182700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113440 con 0x7fcad4108610 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.619+0000 7fcad259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4107cc0 0x7fcad40ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4107cc0 0x7fcad40ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46012/0 (socket says 172.21.15.19:46012) 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 -- 172.21.15.19:0/1915741293 learned_addr learned my addr 172.21.15.19:0/1915741293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad2d9d700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad4103390 0x7fcad40ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad359e700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcad4108610 0x7fcad40ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 -- 172.21.15.19:0/1915741293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcad4108610 msgr2=0x7fcad40ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcad4108610 0x7fcad40ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 -- 172.21.15.19:0/1915741293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad4103390 msgr2=0x7fcad40ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad4103390 0x7fcad40ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 -- 172.21.15.19:0/1915741293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad41a9150 con 0x7fcad4107cc0 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad259c700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4107cc0 0x7fcad40ff6e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcac800d980 tx=0x7fcac800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcabbfff700 1 -- 172.21.15.19:0/1915741293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8017070 con 0x7fcad4107cc0 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcabbfff700 1 -- 172.21.15.19:0/1915741293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac800fd40 con 0x7fcad4107cc0 2024-04-25T10:53:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41a9440 con 0x7fcad4107cc0 2024-04-25T10:53:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcabbfff700 1 -- 172.21.15.19:0/1915741293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac800e3b0 con 0x7fcad4107cc0 2024-04-25T10:53:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.620+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a9980 con 0x7fcad4107cc0 2024-04-25T10:53:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.621+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad4051190 con 0x7fcad4107cc0 2024-04-25T10:53:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.623+0000 7fcabbfff700 1 -- 172.21.15.19:0/1915741293 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcac8010610 con 0x7fcad4107cc0 2024-04-25T10:53:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.623+0000 7fcabbfff700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcabc0afc10 0x7fcabc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.624+0000 7fcad2d9d700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcabc0afc10 0x7fcabc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.624+0000 7fcabbfff700 1 -- 172.21.15.19:0/1915741293 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fcac80de660 con 0x7fcad4107cc0 2024-04-25T10:53:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.624+0000 7fcad2d9d700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcabc0afc10 0x7fcabc0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009990 tx=0x7fcac40036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:28.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.627+0000 7fcabbfff700 1 -- 172.21.15.19:0/1915741293 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcac80a7290 con 0x7fcad4107cc0 2024-04-25T10:53:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.836+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fcad41006d0 con 0x7fcad4107cc0 2024-04-25T10:53:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.836+0000 7fcabbfff700 1 -- 172.21.15.19:0/1915741293 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcac80a6b90 con 0x7fcad4107cc0 2024-04-25T10:53:28.838 INFO:teuthology.orchestra.run.smithi019.stdout:231928234024 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcabc0afc10 msgr2=0x7fcabc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcabc0afc10 0x7fcabc0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009990 tx=0x7fcac40036e0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4107cc0 msgr2=0x7fcad40ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4107cc0 0x7fcad40ff6e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcac800d980 tx=0x7fcac800dc90 comp rx=0 tx=0).stop 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 shutdown_connections 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcabc0afc10 0x7fcabc0b2120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcad4108610 0x7fcad40ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad4103390 0x7fcad40ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 --2- 172.21.15.19:0/1915741293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcad4107cc0 0x7fcad40ff6e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 >> 172.21.15.19:0/1915741293 conn(0x7fcad40fecc0 msgr2=0x7fcad4075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 shutdown_connections 2024-04-25T10:53:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:28.839+0000 7fcad9182700 1 -- 172.21.15.19:0/1915741293 wait complete. 2024-04-25T10:53:29.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:29 smithi019 ceph-mon[29209]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:29.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1915741293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T10:53:29.399 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037790 got 257698037795 for osd.9 2024-04-25T10:53:29.399 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:29.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:29 smithi169 ceph-mon[33049]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:29.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1915741293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T10:53:29.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:29 smithi112 ceph-mon[32799]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:29.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:29 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1915741293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 -- 172.21.15.19:0/227725498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8108610 msgr2=0x7f48c810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 --2- 172.21.15.19:0/227725498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8108610 0x7f48c810ff70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b0d0 tx=0x7f48b800b3e0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 -- 172.21.15.19:0/227725498 shutdown_connections 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 --2- 172.21.15.19:0/227725498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8108610 0x7f48c810ff70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 --2- 172.21.15.19:0/227725498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48c8103390 0x7f48c81037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 --2- 172.21.15.19:0/227725498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c8107cc0 0x7f48c81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 -- 172.21.15.19:0/227725498 >> 172.21.15.19:0/227725498 conn(0x7f48c80fecc0 msgr2=0x7f48c81010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 -- 172.21.15.19:0/227725498 shutdown_connections 2024-04-25T10:53:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.586+0000 7f48cf42c700 1 -- 172.21.15.19:0/227725498 wait complete. 2024-04-25T10:53:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 Processor -- start 2024-04-25T10:53:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 -- start start 2024-04-25T10:53:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48c8103390 0x7f48c81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8107cc0 0x7f48c81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c8108610 0x7f48c819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8113240 con 0x7f48c8107cc0 2024-04-25T10:53:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81130c0 con 0x7f48c8103390 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cf42c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81133c0 con 0x7f48c8108610 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cc9c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8107cc0 0x7f48c81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cc9c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8107cc0 0x7f48c81a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46034/0 (socket says 172.21.15.19:46034) 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cc9c7700 1 -- 172.21.15.19:0/1095834676 learned_addr learned my addr 172.21.15.19:0/1095834676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cd9c9700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c8108610 0x7f48c819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.587+0000 7f48cd1c8700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48c8103390 0x7f48c81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cc9c7700 1 -- 172.21.15.19:0/1095834676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c8108610 msgr2=0x7f48c819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cc9c7700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c8108610 0x7f48c819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cc9c7700 1 -- 172.21.15.19:0/1095834676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48c8103390 msgr2=0x7f48c81a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cc9c7700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48c8103390 0x7f48c81a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cc9c7700 1 -- 172.21.15.19:0/1095834676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c819b160 con 0x7f48c8107cc0 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cc9c7700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8107cc0 0x7f48c81a0dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d980 tx=0x7f48bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48c67fc700 1 -- 172.21.15.19:0/1095834676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc017070 con 0x7f48c8107cc0 2024-04-25T10:53:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48c67fc700 1 -- 172.21.15.19:0/1095834676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48bc00fd40 con 0x7f48c8107cc0 2024-04-25T10:53:29.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48c67fc700 1 -- 172.21.15.19:0/1095834676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00e3b0 con 0x7f48c8107cc0 2024-04-25T10:53:29.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c819b450 con 0x7f48c8107cc0 2024-04-25T10:53:29.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.588+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c81a7720 con 0x7f48c8107cc0 2024-04-25T10:53:29.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.589+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c8051190 con 0x7f48c8107cc0 2024-04-25T10:53:29.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.591+0000 7f48c67fc700 1 -- 172.21.15.19:0/1095834676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f48bc010610 con 0x7f48c8107cc0 2024-04-25T10:53:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.592+0000 7f48c67fc700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f48b00afc90 0x7f48b00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.592+0000 7f48cd1c8700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f48b00afc90 0x7f48b00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.592+0000 7f48c67fc700 1 -- 172.21.15.19:0/1095834676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f48bc0de600 con 0x7f48c8107cc0 2024-04-25T10:53:29.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.592+0000 7f48cd1c8700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f48b00afc90 0x7f48b00b21a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48c819bbf0 tx=0x7f48c00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.596+0000 7f48c67fc700 1 -- 172.21.15.19:0/1095834676 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48bc0a7230 con 0x7f48c8107cc0 2024-04-25T10:53:29.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.812+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f48c802e510 con 0x7f48c8107cc0 2024-04-25T10:53:29.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.813+0000 7f48c67fc700 1 -- 172.21.15.19:0/1095834676 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f48bc0a6b30 con 0x7f48c8107cc0 2024-04-25T10:53:29.814 INFO:teuthology.orchestra.run.smithi019.stdout:184683593778 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f48b00afc90 msgr2=0x7f48b00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f48b00afc90 0x7f48b00b21a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48c819bbf0 tx=0x7f48c00073d0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8107cc0 msgr2=0x7f48c81a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8107cc0 0x7f48c81a0dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d980 tx=0x7f48bc00dc90 comp rx=0 tx=0).stop 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 shutdown_connections 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f48b00afc90 0x7f48b00b21a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c8108610 0x7f48c819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48c8103390 0x7f48c81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 --2- 172.21.15.19:0/1095834676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48c8107cc0 0x7f48c81a0dd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 >> 172.21.15.19:0/1095834676 conn(0x7f48c80fecc0 msgr2=0x7f48c8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 shutdown_connections 2024-04-25T10:53:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:29.815+0000 7f48cf42c700 1 -- 172.21.15.19:0/1095834676 wait complete. 2024-04-25T10:53:30.481 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1095834676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-25T10:53:30.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:30 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-25T10:53:30.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1095834676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T10:53:30.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T10:53:30.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T10:53:30.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T10:53:30.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T10:53:30.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-04-25T10:53:30.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T10:53:30.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-25T10:53:30.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:30 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1095834676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-25T10:53:30.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:30 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-25T10:53:30.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234020 got 231928234024 for osd.8 2024-04-25T10:53:30.597 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 -- 172.21.15.19:0/4136163230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 msgr2=0x7fd668108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 --2- 172.21.15.19:0/4136163230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd668108110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd6500099c0 tx=0x7fd650009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 -- 172.21.15.19:0/4136163230 shutdown_connections 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 --2- 172.21.15.19:0/4136163230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd668108650 0x7fd66810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 --2- 172.21.15.19:0/4136163230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668103420 0x7fd668103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 --2- 172.21.15.19:0/4136163230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd668108110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 -- 172.21.15.19:0/4136163230 >> 172.21.15.19:0/4136163230 conn(0x7fd668078cd0 msgr2=0x7fd668079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 -- 172.21.15.19:0/4136163230 shutdown_connections 2024-04-25T10:53:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.185+0000 7fd66e666700 1 -- 172.21.15.19:0/4136163230 wait complete. 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 Processor -- start 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 -- start start 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd668103420 0x7fd66819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd66819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668108650 0x7fd66819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd668113260 con 0x7fd668107d00 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6681130e0 con 0x7fd668108650 2024-04-25T10:53:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66e666700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6681133e0 con 0x7fd668103420 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd65ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd66819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd65ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd66819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46048/0 (socket says 172.21.15.19:46048) 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd65ffff700 1 -- 172.21.15.19:0/3010397845 learned_addr learned my addr 172.21.15.19:0/3010397845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd66cc03700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668108650 0x7fd66819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.186+0000 7fd667fff700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd668103420 0x7fd66819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd65ffff700 1 -- 172.21.15.19:0/3010397845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd668103420 msgr2=0x7fd66819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd65ffff700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd668103420 0x7fd66819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd65ffff700 1 -- 172.21.15.19:0/3010397845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668108650 msgr2=0x7fd66819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd65ffff700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668108650 0x7fd66819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd65ffff700 1 -- 172.21.15.19:0/3010397845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6681a6e90 con 0x7fd668107d00 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd65ffff700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd66819c080 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd650006010 tx=0x7fd6500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd665ffb700 1 -- 172.21.15.19:0/3010397845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650027070 con 0x7fd668107d00 2024-04-25T10:53:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6681a7120 con 0x7fd668107d00 2024-04-25T10:53:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd665ffb700 1 -- 172.21.15.19:0/3010397845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd650003f20 con 0x7fd668107d00 2024-04-25T10:53:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd665ffb700 1 -- 172.21.15.19:0/3010397845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650022c60 con 0x7fd668107d00 2024-04-25T10:53:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.187+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681a7570 con 0x7fd668107d00 2024-04-25T10:53:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.188+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd668051190 con 0x7fd668107d00 2024-04-25T10:53:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.190+0000 7fd665ffb700 1 -- 172.21.15.19:0/3010397845 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd65002a070 con 0x7fd668107d00 2024-04-25T10:53:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.191+0000 7fd665ffb700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6540afc10 0x7fd6540b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.191+0000 7fd667fff700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6540afc10 0x7fd6540b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.191+0000 7fd665ffb700 1 -- 172.21.15.19:0/3010397845 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fd650022e00 con 0x7fd668107d00 2024-04-25T10:53:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.191+0000 7fd667fff700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6540afc10 0x7fd6540b2120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd658005950 tx=0x7fd65800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:31.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.196+0000 7fd665ffb700 1 -- 172.21.15.19:0/3010397845 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd650013070 con 0x7fd668107d00 2024-04-25T10:53:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.416+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd66802e510 con 0x7fd668107d00 2024-04-25T10:53:31.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:31.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T10:53:31.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T10:53:31.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T10:53:31.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T10:53:31.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-04-25T10:53:31.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T10:53:31.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-25T10:53:31.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-25T10:53:31.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:31 smithi019 ceph-mon[29209]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T10:53:31.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.416+0000 7fd665ffb700 1 -- 172.21.15.19:0/3010397845 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6500a8700 con 0x7fd668107d00 2024-04-25T10:53:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:55834574922 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6540afc10 msgr2=0x7fd6540b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6540afc10 0x7fd6540b2120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd658005950 tx=0x7fd65800b410 comp rx=0 tx=0).stop 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 msgr2=0x7fd66819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd66819c080 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd650006010 tx=0x7fd6500047b0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 shutdown_connections 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd6540afc10 0x7fd6540b2120 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668108650 0x7fd66819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd668103420 0x7fd66819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 --2- 172.21.15.19:0/3010397845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd668107d00 0x7fd66819c080 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 >> 172.21.15.19:0/3010397845 conn(0x7fd668078cd0 msgr2=0x7fd668100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:31.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 shutdown_connections 2024-04-25T10:53:31.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.420+0000 7fd66e666700 1 -- 172.21.15.19:0/3010397845 wait complete. 2024-04-25T10:53:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T10:53:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T10:53:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T10:53:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T10:53:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-04-25T10:53:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T10:53:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-25T10:53:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-25T10:53:31.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:31 smithi169 ceph-mon[33049]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T10:53:31.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:31.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T10:53:31.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T10:53:31.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T10:53:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T10:53:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-04-25T10:53:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T10:53:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-25T10:53:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-25T10:53:31.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:31 smithi112 ceph-mon[32799]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T10:53:31.944 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 -- 172.21.15.19:0/2966420589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd0102860 msgr2=0x7f4dd0102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/2966420589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd0102860 0x7f4dd0102c70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00098f0 tx=0x7f4dc0009c00 comp rx=0 tx=0).stop 2024-04-25T10:53:31.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 -- 172.21.15.19:0/2966420589 shutdown_connections 2024-04-25T10:53:31.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/2966420589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd01031b0 0x7f4dd010f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/2966420589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd0108890 0x7f4dd0108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/2966420589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd0102860 0x7f4dd0102c70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 -- 172.21.15.19:0/2966420589 >> 172.21.15.19:0/2966420589 conn(0x7f4dd00fe3f0 msgr2=0x7f4dd0100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:31.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 -- 172.21.15.19:0/2966420589 shutdown_connections 2024-04-25T10:53:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.975+0000 7f4dd7c99700 1 -- 172.21.15.19:0/2966420589 wait complete. 2024-04-25T10:53:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 Processor -- start 2024-04-25T10:53:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 -- start start 2024-04-25T10:53:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd0102860 0x7f4dd01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd01031b0 0x7f4dd0199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd0108890 0x7f4dd01a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0114440 con 0x7f4dd0108890 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01142c0 con 0x7f4dd01031b0 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd5234700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd01031b0 0x7f4dd0199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd5a35700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd0102860 0x7f4dd01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd5234700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd01031b0 0x7f4dd0199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:34522/0 (socket says 172.21.15.19:34522) 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd5a35700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd0102860 0x7f4dd01998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:40160/0 (socket says 172.21.15.19:40160) 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd5234700 1 -- 172.21.15.19:0/1252509982 learned_addr learned my addr 172.21.15.19:0/1252509982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd6236700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd0108890 0x7f4dd01a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.976+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01145c0 con 0x7f4dd0102860 2024-04-25T10:53:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd5234700 1 -- 172.21.15.19:0/1252509982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd0102860 msgr2=0x7f4dd01998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd5234700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd0102860 0x7f4dd01998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd5234700 1 -- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd0108890 msgr2=0x7f4dd01a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd5234700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd0108890 0x7f4dd01a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd5234700 1 -- 172.21.15.19:0/1252509982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd01a0e50 con 0x7f4dd01031b0 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd5234700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd01031b0 0x7f4dd0199e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000fa00 tx=0x7f4dc000fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dc6ffd700 1 -- 172.21.15.19:0/1252509982 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00056b0 con 0x7f4dd01031b0 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dc6ffd700 1 -- 172.21.15.19:0/1252509982 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc000ba10 con 0x7f4dd01031b0 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dc6ffd700 1 -- 172.21.15.19:0/1252509982 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0021440 con 0x7f4dd01031b0 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd01a10e0 con 0x7f4dd01031b0 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.977+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01a1570 con 0x7f4dd01031b0 2024-04-25T10:53:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.978+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd0103fa0 con 0x7f4dd01031b0 2024-04-25T10:53:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.980+0000 7f4dc6ffd700 1 -- 172.21.15.19:0/1252509982 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4dc0003680 con 0x7f4dd01031b0 2024-04-25T10:53:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.981+0000 7f4dc6ffd700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4dbc0afc40 0x7f4dbc0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.981+0000 7f4dd5a35700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4dbc0afc40 0x7f4dbc0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.981+0000 7f4dc6ffd700 1 -- 172.21.15.19:0/1252509982 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f4dc00dfdd0 con 0x7f4dd01031b0 2024-04-25T10:53:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.982+0000 7f4dd5a35700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4dbc0afc40 0x7f4dbc0b2150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4dd01039a0 tx=0x7f4dcc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:31.985+0000 7f4dc6ffd700 1 -- 172.21.15.19:0/1252509982 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dc0013030 con 0x7f4dd01031b0 2024-04-25T10:53:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.196+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4dd010f360 con 0x7f4dd01031b0 2024-04-25T10:53:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.196+0000 7f4dc6ffd700 1 -- 172.21.15.19:0/1252509982 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4dc00a88f0 con 0x7f4dd01031b0 2024-04-25T10:53:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:137438953532 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4dbc0afc40 msgr2=0x7f4dbc0b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4dbc0afc40 0x7f4dbc0b2150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4dd01039a0 tx=0x7f4dcc009310 comp rx=0 tx=0).stop 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd01031b0 msgr2=0x7f4dd0199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd01031b0 0x7f4dd0199e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000fa00 tx=0x7f4dc000fa30 comp rx=0 tx=0).stop 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 shutdown_connections 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4dbc0afc40 0x7f4dbc0b2150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd0108890 0x7f4dd01a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd0102860 0x7f4dd01998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 --2- 172.21.15.19:0/1252509982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd01031b0 0x7f4dd0199e50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 >> 172.21.15.19:0/1252509982 conn(0x7f4dd00fe3f0 msgr2=0x7f4dd01008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 shutdown_connections 2024-04-25T10:53:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.199+0000 7f4dd7c99700 1 -- 172.21.15.19:0/1252509982 wait complete. 2024-04-25T10:53:32.214 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:32 smithi169 ceph-mon[33049]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T10:53:32.214 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:32 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:53:32.214 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:32 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3010397845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T10:53:32.242 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:32 smithi112 ceph-mon[32799]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T10:53:32.243 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:32 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:53:32.243 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:32 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3010397845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T10:53:32.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 -- 172.21.15.19:0/474344194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc1052f0 msgr2=0x7fadbc105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:32.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 --2- 172.21.15.19:0/474344194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc1052f0 0x7fadbc105700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009990 tx=0x7fadb8009ca0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 -- 172.21.15.19:0/474344194 shutdown_connections 2024-04-25T10:53:32.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 --2- 172.21.15.19:0/474344194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadbc10e760 0x7fadbc110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 --2- 172.21.15.19:0/474344194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc1052f0 0x7fadbc105700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 --2- 172.21.15.19:0/474344194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadbc105c40 0x7fadbc10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 -- 172.21.15.19:0/474344194 >> 172.21.15.19:0/474344194 conn(0x7fadbc0fec30 msgr2=0x7fadbc101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:32.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 -- 172.21.15.19:0/474344194 shutdown_connections 2024-04-25T10:53:32.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.312+0000 7fadc3304700 1 -- 172.21.15.19:0/474344194 wait complete. 2024-04-25T10:53:32.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.313+0000 7fadc3304700 1 Processor -- start 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.313+0000 7fadc3304700 1 -- start start 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc3304700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadbc1052f0 0x7fadbc1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc3304700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadbc105c40 0x7fadbc1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc3304700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc10e760 0x7fadbc19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc3304700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1142b0 con 0x7fadbc10e760 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc3304700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc114130 con 0x7fadbc105c40 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc3304700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc114430 con 0x7fadbc1052f0 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc10e760 0x7fadbc19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc10e760 0x7fadbc19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46088/0 (socket says 172.21.15.19:46088) 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 -- 172.21.15.19:0/3485964317 learned_addr learned my addr 172.21.15.19:0/3485964317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc089f700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadbc105c40 0x7fadbc1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc10a0700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadbc1052f0 0x7fadbc1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 -- 172.21.15.19:0/3485964317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadbc1052f0 msgr2=0x7fadbc1a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadbc1052f0 0x7fadbc1a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 -- 172.21.15.19:0/3485964317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadbc105c40 msgr2=0x7fadbc1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadbc105c40 0x7fadbc1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 -- 172.21.15.19:0/3485964317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc19b120 con 0x7fadbc10e760 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc10a0700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadbc1052f0 0x7fadbc1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.314+0000 7fadc18a1700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc10e760 0x7fadbc19aa70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadb400ef30 tx=0x7fadb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.315+0000 7fadb27fc700 1 -- 172.21.15.19:0/3485964317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40092e0 con 0x7fadbc10e760 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.315+0000 7fadb27fc700 1 -- 172.21.15.19:0/3485964317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb4004bb0 con 0x7fadbc10e760 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.315+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc19b3b0 con 0x7fadbc10e760 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.315+0000 7fadb27fc700 1 -- 172.21.15.19:0/3485964317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4004280 con 0x7fadbc10e760 2024-04-25T10:53:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.315+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc19b8a0 con 0x7fadbc10e760 2024-04-25T10:53:32.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.316+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadbc051190 con 0x7fadbc10e760 2024-04-25T10:53:32.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.317+0000 7fadb27fc700 1 -- 172.21.15.19:0/3485964317 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fadb401b030 con 0x7fadbc10e760 2024-04-25T10:53:32.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.318+0000 7fadb27fc700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fada80afc10 0x7fada80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:32.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.318+0000 7fadc10a0700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fada80afc10 0x7fada80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:32.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.318+0000 7fadb27fc700 1 -- 172.21.15.19:0/3485964317 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fadb40df210 con 0x7fadbc10e760 2024-04-25T10:53:32.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.319+0000 7fadc10a0700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fada80afc10 0x7fada80b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009750 tx=0x7fadb8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:32.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.322+0000 7fadb27fc700 1 -- 172.21.15.19:0/3485964317 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadbc19b8a0 con 0x7fadbc10e760 2024-04-25T10:53:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.531+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fadbc02e510 con 0x7fadbc10e760 2024-04-25T10:53:32.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:32 smithi019 ceph-mon[29209]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T10:53:32.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:32 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:53:32.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:32 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3010397845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T10:53:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.531+0000 7fadb27fc700 1 -- 172.21.15.19:0/3485964317 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fadb4022020 con 0x7fadbc10e760 2024-04-25T10:53:32.534 INFO:teuthology.orchestra.run.smithi019.stdout:309237645337 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fada80afc10 msgr2=0x7fada80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fada80afc10 0x7fada80b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009750 tx=0x7fadb8005fb0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc10e760 msgr2=0x7fadbc19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc10e760 0x7fadbc19aa70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadb400ef30 tx=0x7fadb400c3a0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 shutdown_connections 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fada80afc10 0x7fada80b2120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadbc10e760 0x7fadbc19aa70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadbc1052f0 0x7fadbc1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 --2- 172.21.15.19:0/3485964317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadbc105c40 0x7fadbc1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 >> 172.21.15.19:0/3485964317 conn(0x7fadbc0fec30 msgr2=0x7fadbc0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 shutdown_connections 2024-04-25T10:53:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:32.534+0000 7fadc3304700 1 -- 172.21.15.19:0/3485964317 wait complete. 2024-04-25T10:53:33.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: pgmap v310: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:33.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T10:53:33.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1252509982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T10:53:33.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3485964317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T10:53:33.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.244 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.244 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:33 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.271 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593775 got 184683593778 for osd.6 2024-04-25T10:53:33.271 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:33.296 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:33.296 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:33.297 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: pgmap v310: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T10:53:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1252509982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T10:53:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3485964317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T10:53:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:33 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: pgmap v310: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1252509982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3485964317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:33 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:33.688 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574922 for osd.1 2024-04-25T10:53:33.689 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:34.080 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645332 got 309237645337 for osd.11 2024-04-25T10:53:34.080 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:35.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:35 smithi019 ceph-mon[29209]: pgmap v311: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:35.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:35 smithi169 ceph-mon[33049]: pgmap v311: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:35.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:35 smithi112 ceph-mon[32799]: pgmap v311: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:35.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953525 got 137438953532 for osd.4 2024-04-25T10:53:35.829 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:36.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:36 smithi019 ceph-mon[29209]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-04-25T10:53:36.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:36 smithi019 ceph-mon[29209]: Cluster is now healthy 2024-04-25T10:53:36.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:36 smithi169 ceph-mon[33049]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-04-25T10:53:36.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:36 smithi169 ceph-mon[33049]: Cluster is now healthy 2024-04-25T10:53:36.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:36 smithi112 ceph-mon[32799]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-04-25T10:53:36.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:36 smithi112 ceph-mon[32799]: Cluster is now healthy 2024-04-25T10:53:37.467 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:37 smithi019 ceph-mon[29209]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:37 smithi169 ceph-mon[33049]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:37 smithi112 ceph-mon[32799]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 -- 172.21.15.19:0/1218852906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 msgr2=0x7fc2ec10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/1218852906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec10ff40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400b0d0 tx=0x7fc2e400b3e0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 -- 172.21.15.19:0/1218852906 shutdown_connections 2024-04-25T10:53:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/1218852906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec10ff40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/1218852906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ec103360 0x7fc2ec103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/1218852906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec107c90 0x7fc2ec1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 -- 172.21.15.19:0/1218852906 >> 172.21.15.19:0/1218852906 conn(0x7fc2ec0fecb0 msgr2=0x7fc2ec1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.740+0000 7fc2f1f43700 1 -- 172.21.15.19:0/1218852906 shutdown_connections 2024-04-25T10:53:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.741+0000 7fc2f1f43700 1 -- 172.21.15.19:0/1218852906 wait complete. 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.741+0000 7fc2f1f43700 1 Processor -- start 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.741+0000 7fc2f1f43700 1 -- start start 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2f1f43700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec103360 0x7fc2ec1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2f1f43700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ec107c90 0x7fc2ec1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2f1f43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2f1f43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec113210 con 0x7fc2ec1085e0 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2f1f43700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec113090 con 0x7fc2ec103360 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2f1f43700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec113390 con 0x7fc2ec107c90 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44574/0 (socket says 172.21.15.19:44574) 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 -- 172.21.15.19:0/2031593946 learned_addr learned my addr 172.21.15.19:0/2031593946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2eaffd700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ec107c90 0x7fc2ec1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2eb7fe700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec103360 0x7fc2ec1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 -- 172.21.15.19:0/2031593946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ec107c90 msgr2=0x7fc2ec1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ec107c90 0x7fc2ec1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 -- 172.21.15.19:0/2031593946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec103360 msgr2=0x7fc2ec1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec103360 0x7fc2ec1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.742+0000 7fc2ebfff700 1 -- 172.21.15.19:0/2031593946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec19b0f0 con 0x7fc2ec1085e0 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.743+0000 7fc2ebfff700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec19aa40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40095c0 tx=0x7fc2e4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.743+0000 7fc2e8ff9700 1 -- 172.21.15.19:0/2031593946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e401f070 con 0x7fc2ec1085e0 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.743+0000 7fc2e8ff9700 1 -- 172.21.15.19:0/2031593946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e4007e20 con 0x7fc2ec1085e0 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.743+0000 7fc2e8ff9700 1 -- 172.21.15.19:0/2031593946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e401b4a0 con 0x7fc2ec1085e0 2024-04-25T10:53:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.743+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec19b380 con 0x7fc2ec1085e0 2024-04-25T10:53:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.743+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec19b860 con 0x7fc2ec1085e0 2024-04-25T10:53:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.744+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ec051190 con 0x7fc2ec1085e0 2024-04-25T10:53:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.746+0000 7fc2e8ff9700 1 -- 172.21.15.19:0/2031593946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc2e4022070 con 0x7fc2ec1085e0 2024-04-25T10:53:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.747+0000 7fc2e8ff9700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc2d40afc10 0x7fc2d40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.747+0000 7fc2eb7fe700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc2d40afc10 0x7fc2d40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.747+0000 7fc2e8ff9700 1 -- 172.21.15.19:0/2031593946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fc2e400f020 con 0x7fc2ec1085e0 2024-04-25T10:53:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.747+0000 7fc2eb7fe700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc2d40afc10 0x7fc2d40b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec19bb80 tx=0x7fc2dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.751+0000 7fc2e8ff9700 1 -- 172.21.15.19:0/2031593946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2e4029020 con 0x7fc2ec1085e0 2024-04-25T10:53:38.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.962+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc2ec1a76b0 con 0x7fc2ec1085e0 2024-04-25T10:53:38.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.963+0000 7fc2e8ff9700 1 -- 172.21.15.19:0/2031593946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc2e40a8ee0 con 0x7fc2ec1085e0 2024-04-25T10:53:38.964 INFO:teuthology.orchestra.run.smithi019.stdout:283467841567 2024-04-25T10:53:38.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc2d40afc10 msgr2=0x7fc2d40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:38.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc2d40afc10 0x7fc2d40b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec19bb80 tx=0x7fc2dc00b410 comp rx=0 tx=0).stop 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 msgr2=0x7fc2ec19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec19aa40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40095c0 tx=0x7fc2e4007a20 comp rx=0 tx=0).stop 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 shutdown_connections 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc2d40afc10 0x7fc2d40b2120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ec1085e0 0x7fc2ec19aa40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec103360 0x7fc2ec1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 --2- 172.21.15.19:0/2031593946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ec107c90 0x7fc2ec1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 >> 172.21.15.19:0/2031593946 conn(0x7fc2ec0fecb0 msgr2=0x7fc2ec107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 shutdown_connections 2024-04-25T10:53:38.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:38.965+0000 7fc2f1f43700 1 -- 172.21.15.19:0/2031593946 wait complete. 2024-04-25T10:53:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:39 smithi169 ceph-mon[33049]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:39 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2031593946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T10:53:39.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:39 smithi019 ceph-mon[29209]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:39.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:39 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2031593946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T10:53:39.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.554+0000 7f8bdbe93700 1 -- 172.21.15.19:0/1634935615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4107cc0 msgr2=0x7f8bd41080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.554+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/1634935615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4107cc0 0x7f8bd41080d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40099c0 tx=0x7f8bc4009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.554+0000 7f8bdbe93700 1 -- 172.21.15.19:0/1634935615 shutdown_connections 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.554+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/1634935615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd4108610 0x7f8bd410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.554+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/1634935615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd4103390 0x7f8bd41037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.554+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/1634935615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4107cc0 0x7f8bd41080d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.554+0000 7f8bdbe93700 1 -- 172.21.15.19:0/1634935615 >> 172.21.15.19:0/1634935615 conn(0x7f8bd40fec90 msgr2=0x7f8bd41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.555+0000 7f8bdbe93700 1 -- 172.21.15.19:0/1634935615 shutdown_connections 2024-04-25T10:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.555+0000 7f8bdbe93700 1 -- 172.21.15.19:0/1634935615 wait complete. 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 Processor -- start 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 -- start start 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4103390 0x7f8bd4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd4107cc0 0x7f8bd4199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd4108610 0x7f8bd419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4113220 con 0x7f8bd4103390 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41130a0 con 0x7f8bd4108610 2024-04-25T10:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bdbe93700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41133a0 con 0x7f8bd4107cc0 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bd9c2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4103390 0x7f8bd4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bd9c2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4103390 0x7f8bd4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44584/0 (socket says 172.21.15.19:44584) 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bd9c2f700 1 -- 172.21.15.19:0/681483783 learned_addr learned my addr 172.21.15.19:0/681483783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.556+0000 7f8bd942e700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd4107cc0 0x7f8bd4199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bda430700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd4108610 0x7f8bd419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bd9c2f700 1 -- 172.21.15.19:0/681483783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd4107cc0 msgr2=0x7f8bd4199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bd9c2f700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd4107cc0 0x7f8bd4199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bd9c2f700 1 -- 172.21.15.19:0/681483783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd4108610 msgr2=0x7f8bd419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bd9c2f700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd4108610 0x7f8bd419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bd9c2f700 1 -- 172.21.15.19:0/681483783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd41a9070 con 0x7f8bd4103390 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bd9c2f700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4103390 0x7f8bd4199970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000d950 tx=0x7f8bd000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bcaffd700 1 -- 172.21.15.19:0/681483783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0016070 con 0x7f8bd4103390 2024-04-25T10:53:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bcaffd700 1 -- 172.21.15.19:0/681483783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd000fd40 con 0x7f8bd4103390 2024-04-25T10:53:39.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd41a9360 con 0x7f8bd4103390 2024-04-25T10:53:39.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bcaffd700 1 -- 172.21.15.19:0/681483783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd000e3b0 con 0x7f8bd4103390 2024-04-25T10:53:39.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.557+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd41a9870 con 0x7f8bd4103390 2024-04-25T10:53:39.562 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:39 smithi112 ceph-mon[32799]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T10:53:39.562 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:39 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2031593946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T10:53:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.558+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd4104840 con 0x7f8bd4103390 2024-04-25T10:53:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.560+0000 7f8bcaffd700 1 -- 172.21.15.19:0/681483783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8bd0019070 con 0x7f8bd4103390 2024-04-25T10:53:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.561+0000 7f8bcaffd700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bc00afc10 0x7f8bc00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.561+0000 7f8bd942e700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bc00afc10 0x7f8bc00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.561+0000 7f8bcaffd700 1 -- 172.21.15.19:0/681483783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f8bd00de870 con 0x7f8bd4103390 2024-04-25T10:53:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.561+0000 7f8bd942e700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bc00afc10 0x7f8bc00b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006010 tx=0x7f8bc4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.565+0000 7f8bcaffd700 1 -- 172.21.15.19:0/681483783 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bd00a72a0 con 0x7f8bd4103390 2024-04-25T10:53:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.777+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8bd402e510 con 0x7f8bd4103390 2024-04-25T10:53:39.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.777+0000 7f8bcaffd700 1 -- 172.21.15.19:0/681483783 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8bd00a6ba0 con 0x7f8bd4103390 2024-04-25T10:53:39.779 INFO:teuthology.orchestra.run.smithi019.stdout:77309411400 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bc00afc10 msgr2=0x7f8bc00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bc00afc10 0x7f8bc00b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006010 tx=0x7f8bc4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4103390 msgr2=0x7f8bd4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4103390 0x7f8bd4199970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000d950 tx=0x7f8bd000dc60 comp rx=0 tx=0).stop 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 shutdown_connections 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8bc00afc10 0x7f8bc00b2120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd4108610 0x7f8bd419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd4103390 0x7f8bd4199970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 --2- 172.21.15.19:0/681483783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd4107cc0 0x7f8bd4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 >> 172.21.15.19:0/681483783 conn(0x7f8bd40fec90 msgr2=0x7f8bd41074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 shutdown_connections 2024-04-25T10:53:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:39.779+0000 7f8bdbe93700 1 -- 172.21.15.19:0/681483783 wait complete. 2024-04-25T10:53:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 -- 172.21.15.19:0/2548688933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105330 msgr2=0x7f99a4105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 --2- 172.21.15.19:0/2548688933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105330 0x7f99a4105740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9994009990 tx=0x7f9994009ca0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 -- 172.21.15.19:0/2548688933 shutdown_connections 2024-04-25T10:53:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 --2- 172.21.15.19:0/2548688933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a410e7a0 0x7f99a4110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 --2- 172.21.15.19:0/2548688933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105330 0x7f99a4105740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 --2- 172.21.15.19:0/2548688933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a4105c80 0x7f99a410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 -- 172.21.15.19:0/2548688933 >> 172.21.15.19:0/2548688933 conn(0x7f99a40fec90 msgr2=0x7f99a41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 -- 172.21.15.19:0/2548688933 shutdown_connections 2024-04-25T10:53:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.059+0000 7f99aa227700 1 -- 172.21.15.19:0/2548688933 wait complete. 2024-04-25T10:53:40.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 Processor -- start 2024-04-25T10:53:40.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 -- start start 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a4105330 0x7f99a4103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105c80 0x7f99a4103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a410e7a0 0x7f99a4103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41142f0 con 0x7f99a4105c80 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4114170 con 0x7f99a4105330 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99aa227700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4114470 con 0x7f99a410e7a0 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99a2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105c80 0x7f99a4103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99a2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105c80 0x7f99a4103560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44606/0 (socket says 172.21.15.19:44606) 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.060+0000 7f99a2ffd700 1 -- 172.21.15.19:0/2950171870 learned_addr learned my addr 172.21.15.19:0/2950171870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a3fff700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a410e7a0 0x7f99a4103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a37fe700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a4105330 0x7f99a4103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a2ffd700 1 -- 172.21.15.19:0/2950171870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a410e7a0 msgr2=0x7f99a4103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a2ffd700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a410e7a0 0x7f99a4103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a2ffd700 1 -- 172.21.15.19:0/2950171870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a4105330 msgr2=0x7f99a4103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a2ffd700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a4105330 0x7f99a4103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a2ffd700 1 -- 172.21.15.19:0/2950171870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a41a8f70 con 0x7f99a4105c80 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a37fe700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a4105330 0x7f99a4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a3fff700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a410e7a0 0x7f99a4103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a2ffd700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105c80 0x7f99a4103560 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f999800d950 tx=0x7f999800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a0ff9700 1 -- 172.21.15.19:0/2950171870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998016070 con 0x7f99a4105c80 2024-04-25T10:53:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a0ff9700 1 -- 172.21.15.19:0/2950171870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9998010460 con 0x7f99a4105c80 2024-04-25T10:53:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99a0ff9700 1 -- 172.21.15.19:0/2950171870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999800f620 con 0x7f99a4105c80 2024-04-25T10:53:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a41a9260 con 0x7f99a4105c80 2024-04-25T10:53:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.061+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41a9750 con 0x7f99a4105c80 2024-04-25T10:53:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.062+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a4051190 con 0x7f99a4105c80 2024-04-25T10:53:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.064+0000 7f99a0ff9700 1 -- 172.21.15.19:0/2950171870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9998019070 con 0x7f99a4105c80 2024-04-25T10:53:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.065+0000 7f99a0ff9700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f998c0afc90 0x7f998c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.065+0000 7f99a37fe700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f998c0afc90 0x7f998c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.065+0000 7f99a0ff9700 1 -- 172.21.15.19:0/2950171870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f99980de9a0 con 0x7f99a4105c80 2024-04-25T10:53:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.065+0000 7f99a37fe700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f998c0afc90 0x7f998c0b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9994003800 tx=0x7f999401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.068+0000 7f99a0ff9700 1 -- 172.21.15.19:0/2950171870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99980a7430 con 0x7f99a4105c80 2024-04-25T10:53:40.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.284+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f99a402e510 con 0x7f99a4105c80 2024-04-25T10:53:40.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.284+0000 7f99a0ff9700 1 -- 172.21.15.19:0/2950171870 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99980a6d30 con 0x7f99a4105c80 2024-04-25T10:53:40.286 INFO:teuthology.orchestra.run.smithi019.stdout:206158430255 2024-04-25T10:53:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f998c0afc90 msgr2=0x7f998c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f998c0afc90 0x7f998c0b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9994003800 tx=0x7f999401a040 comp rx=0 tx=0).stop 2024-04-25T10:53:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105c80 msgr2=0x7f99a4103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105c80 0x7f99a4103560 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f999800d950 tx=0x7f999800dd10 comp rx=0 tx=0).stop 2024-04-25T10:53:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 shutdown_connections 2024-04-25T10:53:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f998c0afc90 0x7f998c0b21a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a410e7a0 0x7f99a4103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a4105330 0x7f99a4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 --2- 172.21.15.19:0/2950171870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105c80 0x7f99a4103560 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:40.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 >> 172.21.15.19:0/2950171870 conn(0x7f99a40fec90 msgr2=0x7f99a40ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:40.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 shutdown_connections 2024-04-25T10:53:40.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:40.286+0000 7f99aa227700 1 -- 172.21.15.19:0/2950171870 wait complete. 2024-04-25T10:53:40.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:40 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/681483783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T10:53:40.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:40 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/681483783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T10:53:40.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:40 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/681483783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T10:53:41.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:41 smithi019 ceph-mon[29209]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T10:53:41.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:41 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2950171870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T10:53:41.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:41 smithi169 ceph-mon[33049]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T10:53:41.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:41 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2950171870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T10:53:41.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:41 smithi112 ceph-mon[32799]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T10:53:41.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:41 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2950171870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T10:53:41.635 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411400 for osd.2 2024-04-25T10:53:41.635 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 -- 172.21.15.19:0/2448949007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954108dd0 msgr2=0x7fa9541091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2448949007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954108dd0 0x7fa9541091e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa9400099c0 tx=0x7fa940009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 -- 172.21.15.19:0/2448949007 shutdown_connections 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2448949007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa954109720 0x7fa954111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2448949007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9541044a0 0x7fa9541048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2448949007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954108dd0 0x7fa9541091e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 -- 172.21.15.19:0/2448949007 >> 172.21.15.19:0/2448949007 conn(0x7fa9540ffd70 msgr2=0x7fa9541021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 -- 172.21.15.19:0/2448949007 shutdown_connections 2024-04-25T10:53:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 -- 172.21.15.19:0/2448949007 wait complete. 2024-04-25T10:53:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 Processor -- start 2024-04-25T10:53:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.831+0000 7fa95b07c700 1 -- start start 2024-04-25T10:53:41.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa95b07c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9541044a0 0x7fa95419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa95b07c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa954108dd0 0x7fa95419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa95b07c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954109720 0x7fa95419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa95b07c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa954114330 con 0x7fa954109720 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa95b07c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541141b0 con 0x7fa954108dd0 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa95b07c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541144b0 con 0x7fa9541044a0 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954109720 0x7fa95419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954109720 0x7fa95419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44632/0 (socket says 172.21.15.19:44632) 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 -- 172.21.15.19:0/2543059648 learned_addr learned my addr 172.21.15.19:0/2543059648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa958e18700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9541044a0 0x7fa95419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa953fff700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa954108dd0 0x7fa95419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 -- 172.21.15.19:0/2543059648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9541044a0 msgr2=0x7fa95419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9541044a0 0x7fa95419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 -- 172.21.15.19:0/2543059648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa954108dd0 msgr2=0x7fa95419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa954108dd0 0x7fa95419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.832+0000 7fa959619700 1 -- 172.21.15.19:0/2543059648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95419fb80 con 0x7fa954109720 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.833+0000 7fa959619700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954109720 0x7fa95419f590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa94c007ee0 tx=0x7fa94c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.833+0000 7fa951ffb700 1 -- 172.21.15.19:0/2543059648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004bb0 con 0x7fa954109720 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.833+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95419fe70 con 0x7fa954109720 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.833+0000 7fa951ffb700 1 -- 172.21.15.19:0/2543059648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c004d50 con 0x7fa954109720 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.833+0000 7fa951ffb700 1 -- 172.21.15.19:0/2543059648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c005900 con 0x7fa954109720 2024-04-25T10:53:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.833+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a0380 con 0x7fa954109720 2024-04-25T10:53:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.835+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa954051190 con 0x7fa954109720 2024-04-25T10:53:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.836+0000 7fa951ffb700 1 -- 172.21.15.19:0/2543059648 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa94c01a020 con 0x7fa954109720 2024-04-25T10:53:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.837+0000 7fa951ffb700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9440afc10 0x7fa9440b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.837+0000 7fa951ffb700 1 -- 172.21.15.19:0/2543059648 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fa94c0de440 con 0x7fa954109720 2024-04-25T10:53:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.837+0000 7fa958e18700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9440afc10 0x7fa9440b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.837+0000 7fa958e18700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9440afc10 0x7fa9440b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa94800ac50 tx=0x7fa94800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:41.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:41.841+0000 7fa951ffb700 1 -- 172.21.15.19:0/2543059648 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa94c010070 con 0x7fa954109720 2024-04-25T10:53:42.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.048+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa95402e510 con 0x7fa954109720 2024-04-25T10:53:42.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.049+0000 7fa951ffb700 1 -- 172.21.15.19:0/2543059648 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa94c0a6ed0 con 0x7fa954109720 2024-04-25T10:53:42.051 INFO:teuthology.orchestra.run.smithi019.stdout:34359738450 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9440afc10 msgr2=0x7fa9440b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9440afc10 0x7fa9440b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa94800ac50 tx=0x7fa94800a380 comp rx=0 tx=0).stop 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954109720 msgr2=0x7fa95419f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954109720 0x7fa95419f590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa94c007ee0 tx=0x7fa94c00d3b0 comp rx=0 tx=0).stop 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 shutdown_connections 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fa9440afc10 0x7fa9440b2120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954109720 0x7fa95419f590 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9541044a0 0x7fa95419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 --2- 172.21.15.19:0/2543059648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa954108dd0 0x7fa95419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 >> 172.21.15.19:0/2543059648 conn(0x7fa9540ffd70 msgr2=0x7fa95410a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 shutdown_connections 2024-04-25T10:53:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:42.051+0000 7fa95b07c700 1 -- 172.21.15.19:0/2543059648 wait complete. 2024-04-25T10:53:42.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:42 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2543059648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T10:53:42.432 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430248 got 206158430255 for osd.7 2024-04-25T10:53:42.432 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:42 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2543059648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T10:53:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:42 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2543059648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T10:53:42.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841561 got 283467841567 for osd.10 2024-04-25T10:53:42.863 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:43.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:43 smithi019 ceph-mon[29209]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T10:53:43.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738450 for osd.0 2024-04-25T10:53:43.487 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:43 smithi169 ceph-mon[33049]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T10:53:43.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:43 smithi112 ceph-mon[32799]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T10:53:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.017+0000 7fc18145c700 1 -- 172.21.15.19:0/3145053656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 msgr2=0x7fc17c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.017+0000 7fc18145c700 1 --2- 172.21.15.19:0/3145053656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c104870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0099c0 tx=0x7fc16c009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.017+0000 7fc18145c700 1 -- 172.21.15.19:0/3145053656 shutdown_connections 2024-04-25T10:53:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.017+0000 7fc18145c700 1 --2- 172.21.15.19:0/3145053656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c1096e0 0x7fc17c111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.017+0000 7fc18145c700 1 --2- 172.21.15.19:0/3145053656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c104870 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.017+0000 7fc18145c700 1 --2- 172.21.15.19:0/3145053656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c108d90 0x7fc17c1091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.017+0000 7fc18145c700 1 -- 172.21.15.19:0/3145053656 >> 172.21.15.19:0/3145053656 conn(0x7fc17c0ffd40 msgr2=0x7fc17c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.018+0000 7fc18145c700 1 -- 172.21.15.19:0/3145053656 shutdown_connections 2024-04-25T10:53:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.018+0000 7fc18145c700 1 -- 172.21.15.19:0/3145053656 wait complete. 2024-04-25T10:53:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.018+0000 7fc18145c700 1 Processor -- start 2024-04-25T10:53:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc18145c700 1 -- start start 2024-04-25T10:53:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc18145c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc18145c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c108d90 0x7fc17c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc18145c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c1096e0 0x7fc17c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc18145c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c114400 con 0x7fc17c104460 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc18145c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c114280 con 0x7fc17c108d90 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc18145c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c114580 con 0x7fc17c1096e0 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc17affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc17affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59090/0 (socket says 172.21.15.19:59090) 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc17affd700 1 -- 172.21.15.19:0/2342458263 learned_addr learned my addr 172.21.15.19:0/2342458263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.019+0000 7fc17b7fe700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c1096e0 0x7fc17c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc17a7fc700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c108d90 0x7fc17c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc17affd700 1 -- 172.21.15.19:0/2342458263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c1096e0 msgr2=0x7fc17c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc17affd700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c1096e0 0x7fc17c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc17affd700 1 -- 172.21.15.19:0/2342458263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c108d90 msgr2=0x7fc17c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc17affd700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c108d90 0x7fc17c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc17affd700 1 -- 172.21.15.19:0/2342458263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c19fc60 con 0x7fc17c104460 2024-04-25T10:53:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc17affd700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c19ad90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0098e0 tx=0x7fc16c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc163fff700 1 -- 172.21.15.19:0/2342458263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c027070 con 0x7fc17c104460 2024-04-25T10:53:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c19fef0 con 0x7fc17c104460 2024-04-25T10:53:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc163fff700 1 -- 172.21.15.19:0/2342458263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16c003f20 con 0x7fc17c104460 2024-04-25T10:53:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc163fff700 1 -- 172.21.15.19:0/2342458263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c022d70 con 0x7fc17c104460 2024-04-25T10:53:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.020+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c1a7870 con 0x7fc17c104460 2024-04-25T10:53:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.021+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17c105910 con 0x7fc17c104460 2024-04-25T10:53:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.023+0000 7fc163fff700 1 -- 172.21.15.19:0/2342458263 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc16c02a070 con 0x7fc17c104460 2024-04-25T10:53:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.024+0000 7fc163fff700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc1640afc90 0x7fc1640b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.024+0000 7fc17a7fc700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc1640afc90 0x7fc1640b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.024+0000 7fc163fff700 1 -- 172.21.15.19:0/2342458263 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fc16c0e0440 con 0x7fc17c104460 2024-04-25T10:53:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.024+0000 7fc17a7fc700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc1640afc90 0x7fc1640b21a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc17c107460 tx=0x7fc170009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:44.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.028+0000 7fc163fff700 1 -- 172.21.15.19:0/2342458263 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc16c031080 con 0x7fc17c104460 2024-04-25T10:53:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.285+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc17c19c0f0 con 0x7fc17c104460 2024-04-25T10:53:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.286+0000 7fc163fff700 1 -- 172.21.15.19:0/2342458263 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc16c013070 con 0x7fc17c104460 2024-04-25T10:53:44.288 INFO:teuthology.orchestra.run.smithi019.stdout:111669149763 2024-04-25T10:53:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.288+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc1640afc90 msgr2=0x7fc1640b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.288+0000 7fc18145c700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc1640afc90 0x7fc1640b21a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc17c107460 tx=0x7fc170009190 comp rx=0 tx=0).stop 2024-04-25T10:53:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.288+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 msgr2=0x7fc17c19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.288+0000 7fc18145c700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c19ad90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0098e0 tx=0x7fc16c0057c0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 shutdown_connections 2024-04-25T10:53:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc1640afc90 0x7fc1640b21a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c1096e0 0x7fc17c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc17c104460 0x7fc17c19ad90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 --2- 172.21.15.19:0/2342458263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c108d90 0x7fc17c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 >> 172.21.15.19:0/2342458263 conn(0x7fc17c0ffd40 msgr2=0x7fc17c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 shutdown_connections 2024-04-25T10:53:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:44.289+0000 7fc18145c700 1 -- 172.21.15.19:0/2342458263 wait complete. 2024-04-25T10:53:44.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149759 got 111669149763 for osd.3 2024-04-25T10:53:44.989 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:44.989 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T10:53:44.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T10:53:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:45 smithi169 ceph-mon[33049]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T10:53:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:45 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2342458263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T10:53:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:53:45.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:53:45.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:45 smithi169 ceph-mon[33049]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:45 smithi019 ceph-mon[29209]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T10:53:45.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:45 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2342458263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T10:53:45.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:53:45.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:53:45.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:45 smithi019 ceph-mon[29209]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:45 smithi112 ceph-mon[32799]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T10:53:45.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:45 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2342458263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T10:53:45.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:45.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:53:45.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:53:45.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:45 smithi112 ceph-mon[32799]: from='mgr.14150 172.21.15.19:0/1085896310' entity='mgr.x' 2024-04-25T10:53:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 -- 172.21.15.19:0/4244277875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 msgr2=0x7fd924102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 --2- 172.21.15.19:0/4244277875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924102dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd9200099c0 tx=0x7fd920009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 -- 172.21.15.19:0/4244277875 shutdown_connections 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 --2- 172.21.15.19:0/4244277875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924109720 0x7fd924111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 --2- 172.21.15.19:0/4244277875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924102dc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 --2- 172.21.15.19:0/4244277875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd924108dd0 0x7fd9241091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 -- 172.21.15.19:0/4244277875 >> 172.21.15.19:0/4244277875 conn(0x7fd9240fe3f0 msgr2=0x7fd924100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 -- 172.21.15.19:0/4244277875 shutdown_connections 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.895+0000 7fd92c693700 1 -- 172.21.15.19:0/4244277875 wait complete. 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 Processor -- start 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 -- start start 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924103900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924108dd0 0x7fd924103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd924109720 0x7fd924104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd924114350 con 0x7fd9241029b0 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241141d0 con 0x7fd924108dd0 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.896+0000 7fd92c693700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241144d0 con 0x7fd924109720 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924103900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924103900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59110/0 (socket says 172.21.15.19:59110) 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 -- 172.21.15.19:0/2581714885 learned_addr learned my addr 172.21.15.19:0/2581714885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd929c2e700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924108dd0 0x7fd924103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92ac30700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd924109720 0x7fd924104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 -- 172.21.15.19:0/2581714885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd924109720 msgr2=0x7fd924104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd924109720 0x7fd924104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 -- 172.21.15.19:0/2581714885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924108dd0 msgr2=0x7fd924103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924108dd0 0x7fd924103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 -- 172.21.15.19:0/2581714885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9241a9120 con 0x7fd9241029b0 2024-04-25T10:53:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92a42f700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924103900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b580 tx=0x7fd9200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd91b7fe700 1 -- 172.21.15.19:0/2581714885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920027070 con 0x7fd9241029b0 2024-04-25T10:53:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9241a93b0 con 0x7fd9241029b0 2024-04-25T10:53:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd91b7fe700 1 -- 172.21.15.19:0/2581714885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd920003f20 con 0x7fd9241029b0 2024-04-25T10:53:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd91b7fe700 1 -- 172.21.15.19:0/2581714885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920022d10 con 0x7fd9241029b0 2024-04-25T10:53:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.897+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9241a9860 con 0x7fd9241029b0 2024-04-25T10:53:46.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.899+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd924051190 con 0x7fd9241029b0 2024-04-25T10:53:46.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.900+0000 7fd91b7fe700 1 -- 172.21.15.19:0/2581714885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd920022470 con 0x7fd9241029b0 2024-04-25T10:53:46.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.901+0000 7fd91b7fe700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd9100afc10 0x7fd9100b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:46.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.901+0000 7fd929c2e700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd9100afc10 0x7fd9100b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:46.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.901+0000 7fd91b7fe700 1 -- 172.21.15.19:0/2581714885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fd9200dfa00 con 0x7fd9241029b0 2024-04-25T10:53:46.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.901+0000 7fd929c2e700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd9100afc10 0x7fd9100b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9241075c0 tx=0x7fd91400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:46.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:46.906+0000 7fd91b7fe700 1 -- 172.21.15.19:0/2581714885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9200e1e20 con 0x7fd9241029b0 2024-04-25T10:53:47.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.107+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd924104e30 con 0x7fd9100afc10 2024-04-25T10:53:47.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.114+0000 7fd91b7fe700 1 -- 172.21.15.19:0/2581714885 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258627 (secure 0 0 0) 0x7fd924104e30 con 0x7fd9100afc10 2024-04-25T10:53:47.119 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd9100afc10 msgr2=0x7fd9100b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd9100afc10 0x7fd9100b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9241075c0 tx=0x7fd91400b410 comp rx=0 tx=0).stop 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 msgr2=0x7fd924103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924103900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b580 tx=0x7fd9200047b0 comp rx=0 tx=0).stop 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 shutdown_connections 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fd9100afc10 0x7fd9100b2120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd924109720 0x7fd924104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.117+0000 7fd92c693700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9241029b0 0x7fd924103900 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.118+0000 7fd92c693700 1 --2- 172.21.15.19:0/2581714885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924108dd0 0x7fd924103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.118+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 >> 172.21.15.19:0/2581714885 conn(0x7fd9240fe3f0 msgr2=0x7fd9241013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.118+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 shutdown_connections 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:47.118+0000 7fd92c693700 1 -- 172.21.15.19:0/2581714885 wait complete. 2024-04-25T10:53:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:53:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:47 smithi169 ceph-mon[33049]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T10:53:47.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:47 smithi112 ceph-mon[32799]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T10:53:47.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:47 smithi019 ceph-mon[29209]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T10:53:47.763 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-25T10:53:46.012314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":234,"num_read_kb":207,"num_write":200,"num_write_kb":2235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365304,"kb_used_data":11916,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576520,"statfs":{"total":1151940427776,"available":1151566356480,"internally_reserved":0,"allocated":12201984,"data_stored":6891126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150151+0000","last_change":"2024-04-25T10:52:32.474738+0000","last_active":"2024-04-25T10:53:31.150151+0000","last_peered":"2024-04-25T10:53:31.150151+0000","last_clean":"2024-04-25T10:53:31.150151+0000","last_became_active":"2024-04-25T10:52:32.473812+0000","last_became_peered":"2024-04-25T10:52:32.473812+0000","last_unstale":"2024-04-25T10:53:31.150151+0000","last_undegraded":"2024-04-25T10:53:31.150151+0000","last_fullsized":"2024-04-25T10:53:31.150151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:25.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300204+0000","last_change":"2024-04-25T10:52:32.475491+0000","last_active":"2024-04-25T10:53:31.300204+0000","last_peered":"2024-04-25T10:53:31.300204+0000","last_clean":"2024-04-25T10:53:31.300204+0000","last_became_active":"2024-04-25T10:52:32.474164+0000","last_became_peered":"2024-04-25T10:52:32.474164+0000","last_unstale":"2024-04-25T10:53:31.300204+0000","last_undegraded":"2024-04-25T10:53:31.300204+0000","last_fullsized":"2024-04-25T10:53:31.300204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:58.838292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300309+0000","last_change":"2024-04-25T10:52:32.471349+0000","last_active":"2024-04-25T10:53:31.300309+0000","last_peered":"2024-04-25T10:53:31.300309+0000","last_clean":"2024-04-25T10:53:31.300309+0000","last_became_active":"2024-04-25T10:52:32.470972+0000","last_became_peered":"2024-04-25T10:52:32.470972+0000","last_unstale":"2024-04-25T10:53:31.300309+0000","last_undegraded":"2024-04-25T10:53:31.300309+0000","last_fullsized":"2024-04-25T10:53:31.300309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:28.634537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243291+0000","last_change":"2024-04-25T10:52:32.470212+0000","last_active":"2024-04-25T10:53:31.243291+0000","last_peered":"2024-04-25T10:53:31.243291+0000","last_clean":"2024-04-25T10:53:31.243291+0000","last_became_active":"2024-04-25T10:52:32.469642+0000","last_became_peered":"2024-04-25T10:52:32.469642+0000","last_unstale":"2024-04-25T10:53:31.243291+0000","last_undegraded":"2024-04-25T10:53:31.243291+0000","last_fullsized":"2024-04-25T10:53:31.243291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:08.231679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147067+0000","last_change":"2024-04-25T10:52:32.456568+0000","last_active":"2024-04-25T10:53:31.147067+0000","last_peered":"2024-04-25T10:53:31.147067+0000","last_clean":"2024-04-25T10:53:31.147067+0000","last_became_active":"2024-04-25T10:52:32.456051+0000","last_became_peered":"2024-04-25T10:52:32.456051+0000","last_unstale":"2024-04-25T10:53:31.147067+0000","last_undegraded":"2024-04-25T10:53:31.147067+0000","last_fullsized":"2024-04-25T10:53:31.147067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243529+0000","last_change":"2024-04-25T10:52:32.479781+0000","last_active":"2024-04-25T10:53:31.243529+0000","last_peered":"2024-04-25T10:53:31.243529+0000","last_clean":"2024-04-25T10:53:31.243529+0000","last_became_active":"2024-04-25T10:52:32.479213+0000","last_became_peered":"2024-04-25T10:52:32.479213+0000","last_unstale":"2024-04-25T10:53:31.243529+0000","last_undegraded":"2024-04-25T10:53:31.243529+0000","last_fullsized":"2024-04-25T10:53:31.243529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:16.239969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149836+0000","last_change":"2024-04-25T10:52:32.471430+0000","last_active":"2024-04-25T10:53:31.149836+0000","last_peered":"2024-04-25T10:53:31.149836+0000","last_clean":"2024-04-25T10:53:31.149836+0000","last_became_active":"2024-04-25T10:52:32.470518+0000","last_became_peered":"2024-04-25T10:52:32.470518+0000","last_unstale":"2024-04-25T10:53:31.149836+0000","last_undegraded":"2024-04-25T10:53:31.149836+0000","last_fullsized":"2024-04-25T10:53:31.149836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:12.846174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152841+0000","last_change":"2024-04-25T10:52:32.472545+0000","last_active":"2024-04-25T10:53:31.152841+0000","last_peered":"2024-04-25T10:53:31.152841+0000","last_clean":"2024-04-25T10:53:31.152841+0000","last_became_active":"2024-04-25T10:52:32.472151+0000","last_became_peered":"2024-04-25T10:52:32.472151+0000","last_unstale":"2024-04-25T10:53:31.152841+0000","last_undegraded":"2024-04-25T10:53:31.152841+0000","last_fullsized":"2024-04-25T10:53:31.152841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:36.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146931+0000","last_change":"2024-04-25T10:52:32.466223+0000","last_active":"2024-04-25T10:53:31.146931+0000","last_peered":"2024-04-25T10:53:31.146931+0000","last_clean":"2024-04-25T10:53:31.146931+0000","last_became_active":"2024-04-25T10:52:32.465498+0000","last_became_peered":"2024-04-25T10:52:32.465498+0000","last_unstale":"2024-04-25T10:53:31.146931+0000","last_undegraded":"2024-04-25T10:53:31.146931+0000","last_fullsized":"2024-04-25T10:53:31.146931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:25.924450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244273+0000","last_change":"2024-04-25T10:52:32.470601+0000","last_active":"2024-04-25T10:53:31.244273+0000","last_peered":"2024-04-25T10:53:31.244273+0000","last_clean":"2024-04-25T10:53:31.244273+0000","last_became_active":"2024-04-25T10:52:32.468984+0000","last_became_peered":"2024-04-25T10:52:32.468984+0000","last_unstale":"2024-04-25T10:53:31.244273+0000","last_undegraded":"2024-04-25T10:53:31.244273+0000","last_fullsized":"2024-04-25T10:53:31.244273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150337+0000","last_change":"2024-04-25T10:52:32.463907+0000","last_active":"2024-04-25T10:53:31.150337+0000","last_peered":"2024-04-25T10:53:31.150337+0000","last_clean":"2024-04-25T10:53:31.150337+0000","last_became_active":"2024-04-25T10:52:32.463536+0000","last_became_peered":"2024-04-25T10:52:32.463536+0000","last_unstale":"2024-04-25T10:53:31.150337+0000","last_undegraded":"2024-04-25T10:53:31.150337+0000","last_fullsized":"2024-04-25T10:53:31.150337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.503805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243169+0000","last_change":"2024-04-25T10:52:32.473646+0000","last_active":"2024-04-25T10:53:31.243169+0000","last_peered":"2024-04-25T10:53:31.243169+0000","last_clean":"2024-04-25T10:53:31.243169+0000","last_became_active":"2024-04-25T10:52:32.473310+0000","last_became_peered":"2024-04-25T10:52:32.473310+0000","last_unstale":"2024-04-25T10:53:31.243169+0000","last_undegraded":"2024-04-25T10:53:31.243169+0000","last_fullsized":"2024-04-25T10:53:31.243169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.691293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244988+0000","last_change":"2024-04-25T10:53:31.244988+0000","last_active":"2024-04-25T10:53:31.244988+0000","last_peered":"2024-04-25T10:53:31.244988+0000","last_clean":"2024-04-25T10:53:31.244988+0000","last_became_active":"2024-04-25T10:53:31.244535+0000","last_became_peered":"2024-04-25T10:53:31.244535+0000","last_unstale":"2024-04-25T10:53:31.244988+0000","last_undegraded":"2024-04-25T10:53:31.244988+0000","last_fullsized":"2024-04-25T10:53:31.244988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:14.205691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300514+0000","last_change":"2024-04-25T10:52:32.476104+0000","last_active":"2024-04-25T10:53:31.300514+0000","last_peered":"2024-04-25T10:53:31.300514+0000","last_clean":"2024-04-25T10:53:31.300514+0000","last_became_active":"2024-04-25T10:52:32.475342+0000","last_became_peered":"2024-04-25T10:52:32.475342+0000","last_unstale":"2024-04-25T10:53:31.300514+0000","last_undegraded":"2024-04-25T10:53:31.300514+0000","last_fullsized":"2024-04-25T10:53:31.300514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:37.087793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300100+0000","last_change":"2024-04-25T10:52:32.470263+0000","last_active":"2024-04-25T10:53:31.300100+0000","last_peered":"2024-04-25T10:53:31.300100+0000","last_clean":"2024-04-25T10:53:31.300100+0000","last_became_active":"2024-04-25T10:52:32.469873+0000","last_became_peered":"2024-04-25T10:52:32.469873+0000","last_unstale":"2024-04-25T10:53:31.300100+0000","last_undegraded":"2024-04-25T10:53:31.300100+0000","last_fullsized":"2024-04-25T10:53:31.300100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.349598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146088+0000","last_change":"2024-04-25T10:52:32.471851+0000","last_active":"2024-04-25T10:53:31.146088+0000","last_peered":"2024-04-25T10:53:31.146088+0000","last_clean":"2024-04-25T10:53:31.146088+0000","last_became_active":"2024-04-25T10:52:32.471477+0000","last_became_peered":"2024-04-25T10:52:32.471477+0000","last_unstale":"2024-04-25T10:53:31.146088+0000","last_undegraded":"2024-04-25T10:53:31.146088+0000","last_fullsized":"2024-04-25T10:53:31.146088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:59.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150020+0000","last_change":"2024-04-25T10:52:32.472530+0000","last_active":"2024-04-25T10:53:31.150020+0000","last_peered":"2024-04-25T10:53:31.150020+0000","last_clean":"2024-04-25T10:53:31.150020+0000","last_became_active":"2024-04-25T10:52:32.472174+0000","last_became_peered":"2024-04-25T10:52:32.472174+0000","last_unstale":"2024-04-25T10:53:31.150020+0000","last_undegraded":"2024-04-25T10:53:31.150020+0000","last_fullsized":"2024-04-25T10:53:31.150020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:09.473575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150137+0000","last_change":"2024-04-25T10:52:32.473617+0000","last_active":"2024-04-25T10:53:31.150137+0000","last_peered":"2024-04-25T10:53:31.150137+0000","last_clean":"2024-04-25T10:53:31.150137+0000","last_became_active":"2024-04-25T10:52:32.473261+0000","last_became_peered":"2024-04-25T10:52:32.473261+0000","last_unstale":"2024-04-25T10:53:31.150137+0000","last_undegraded":"2024-04-25T10:53:31.150137+0000","last_fullsized":"2024-04-25T10:53:31.150137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:05.568672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242750+0000","last_change":"2024-04-25T10:52:32.466296+0000","last_active":"2024-04-25T10:53:31.242750+0000","last_peered":"2024-04-25T10:53:31.242750+0000","last_clean":"2024-04-25T10:53:31.242750+0000","last_became_active":"2024-04-25T10:52:32.463153+0000","last_became_peered":"2024-04-25T10:52:32.463153+0000","last_unstale":"2024-04-25T10:53:31.242750+0000","last_undegraded":"2024-04-25T10:53:31.242750+0000","last_fullsized":"2024-04-25T10:53:31.242750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150155+0000","last_change":"2024-04-25T10:52:31.701949+0000","last_active":"2024-04-25T10:53:31.150155+0000","last_peered":"2024-04-25T10:53:31.150155+0000","last_clean":"2024-04-25T10:53:31.150155+0000","last_became_active":"2024-04-25T10:52:31.701508+0000","last_became_peered":"2024-04-25T10:52:31.701508+0000","last_unstale":"2024-04-25T10:53:31.150155+0000","last_undegraded":"2024-04-25T10:53:31.150155+0000","last_fullsized":"2024-04-25T10:53:31.150155+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:41.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153836+0000","last_change":"2024-04-25T10:52:32.468871+0000","last_active":"2024-04-25T10:53:31.153836+0000","last_peered":"2024-04-25T10:53:31.153836+0000","last_clean":"2024-04-25T10:53:31.153836+0000","last_became_active":"2024-04-25T10:52:32.468460+0000","last_became_peered":"2024-04-25T10:52:32.468460+0000","last_unstale":"2024-04-25T10:53:31.153836+0000","last_undegraded":"2024-04-25T10:53:31.153836+0000","last_fullsized":"2024-04-25T10:53:31.153836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:29.055639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154101+0000","last_change":"2024-04-25T10:52:31.705068+0000","last_active":"2024-04-25T10:53:31.154101+0000","last_peered":"2024-04-25T10:53:31.154101+0000","last_clean":"2024-04-25T10:53:31.154101+0000","last_became_active":"2024-04-25T10:52:31.704439+0000","last_became_peered":"2024-04-25T10:52:31.704439+0000","last_unstale":"2024-04-25T10:53:31.154101+0000","last_undegraded":"2024-04-25T10:53:31.154101+0000","last_fullsized":"2024-04-25T10:53:31.154101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.347653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300337+0000","last_change":"2024-04-25T10:52:32.476025+0000","last_active":"2024-04-25T10:53:31.300337+0000","last_peered":"2024-04-25T10:53:31.300337+0000","last_clean":"2024-04-25T10:53:31.300337+0000","last_became_active":"2024-04-25T10:52:32.475611+0000","last_became_peered":"2024-04-25T10:52:32.475611+0000","last_unstale":"2024-04-25T10:53:31.300337+0000","last_undegraded":"2024-04-25T10:53:31.300337+0000","last_fullsized":"2024-04-25T10:53:31.300337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:45.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147487+0000","last_change":"2024-04-25T10:52:31.703012+0000","last_active":"2024-04-25T10:53:31.147487+0000","last_peered":"2024-04-25T10:53:31.147487+0000","last_clean":"2024-04-25T10:53:31.147487+0000","last_became_active":"2024-04-25T10:52:31.702689+0000","last_became_peered":"2024-04-25T10:52:31.702689+0000","last_unstale":"2024-04-25T10:53:31.147487+0000","last_undegraded":"2024-04-25T10:53:31.147487+0000","last_fullsized":"2024-04-25T10:53:31.147487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:13.038859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146541+0000","last_change":"2024-04-25T10:52:32.475858+0000","last_active":"2024-04-25T10:53:31.146541+0000","last_peered":"2024-04-25T10:53:31.146541+0000","last_clean":"2024-04-25T10:53:31.146541+0000","last_became_active":"2024-04-25T10:52:32.475486+0000","last_became_peered":"2024-04-25T10:52:32.475486+0000","last_unstale":"2024-04-25T10:53:31.146541+0000","last_undegraded":"2024-04-25T10:53:31.146541+0000","last_fullsized":"2024-04-25T10:53:31.146541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.439208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.155911+0000","last_change":"2024-04-25T10:53:31.155911+0000","last_active":"2024-04-25T10:53:31.155911+0000","last_peered":"2024-04-25T10:53:31.155911+0000","last_clean":"2024-04-25T10:53:31.155911+0000","last_became_active":"2024-04-25T10:53:31.151778+0000","last_became_peered":"2024-04-25T10:53:31.151778+0000","last_unstale":"2024-04-25T10:53:31.155911+0000","last_undegraded":"2024-04-25T10:53:31.155911+0000","last_fullsized":"2024-04-25T10:53:31.155911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:46.936583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152237+0000","last_change":"2024-04-25T10:53:31.152237+0000","last_active":"2024-04-25T10:53:31.152237+0000","last_peered":"2024-04-25T10:53:31.152237+0000","last_clean":"2024-04-25T10:53:31.152237+0000","last_became_active":"2024-04-25T10:53:31.151821+0000","last_became_peered":"2024-04-25T10:53:31.151821+0000","last_unstale":"2024-04-25T10:53:31.152237+0000","last_undegraded":"2024-04-25T10:53:31.152237+0000","last_fullsized":"2024-04-25T10:53:31.152237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:02.453018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148877+0000","last_change":"2024-04-25T10:52:31.699644+0000","last_active":"2024-04-25T10:53:31.148877+0000","last_peered":"2024-04-25T10:53:31.148877+0000","last_clean":"2024-04-25T10:53:31.148877+0000","last_became_active":"2024-04-25T10:52:31.699315+0000","last_became_peered":"2024-04-25T10:52:31.699315+0000","last_unstale":"2024-04-25T10:53:31.148877+0000","last_undegraded":"2024-04-25T10:53:31.148877+0000","last_fullsized":"2024-04-25T10:53:31.148877+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:52.915096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147070+0000","last_change":"2024-04-25T10:52:32.463372+0000","last_active":"2024-04-25T10:53:31.147070+0000","last_peered":"2024-04-25T10:53:31.147070+0000","last_clean":"2024-04-25T10:53:31.147070+0000","last_became_active":"2024-04-25T10:52:32.462926+0000","last_became_peered":"2024-04-25T10:52:32.462926+0000","last_unstale":"2024-04-25T10:53:31.147070+0000","last_undegraded":"2024-04-25T10:53:31.147070+0000","last_fullsized":"2024-04-25T10:53:31.147070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:17.724268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243415+0000","last_change":"2024-04-25T10:52:31.695754+0000","last_active":"2024-04-25T10:53:31.243415+0000","last_peered":"2024-04-25T10:53:31.243415+0000","last_clean":"2024-04-25T10:53:31.243415+0000","last_became_active":"2024-04-25T10:52:31.695381+0000","last_became_peered":"2024-04-25T10:52:31.695381+0000","last_unstale":"2024-04-25T10:53:31.243415+0000","last_undegraded":"2024-04-25T10:53:31.243415+0000","last_fullsized":"2024-04-25T10:53:31.243415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:58.851534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154575+0000","last_change":"2024-04-25T10:53:31.154575+0000","last_active":"2024-04-25T10:53:31.154575+0000","last_peered":"2024-04-25T10:53:31.154575+0000","last_clean":"2024-04-25T10:53:31.154575+0000","last_became_active":"2024-04-25T10:53:31.154145+0000","last_became_peered":"2024-04-25T10:53:31.154145+0000","last_unstale":"2024-04-25T10:53:31.154575+0000","last_undegraded":"2024-04-25T10:53:31.154575+0000","last_fullsized":"2024-04-25T10:53:31.154575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:37.838587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145615+0000","last_change":"2024-04-25T10:52:31.695067+0000","last_active":"2024-04-25T10:53:31.145615+0000","last_peered":"2024-04-25T10:53:31.145615+0000","last_clean":"2024-04-25T10:53:31.145615+0000","last_became_active":"2024-04-25T10:52:31.694648+0000","last_became_peered":"2024-04-25T10:52:31.694648+0000","last_unstale":"2024-04-25T10:53:31.145615+0000","last_undegraded":"2024-04-25T10:53:31.145615+0000","last_fullsized":"2024-04-25T10:53:31.145615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:09.654835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152895+0000","last_change":"2024-04-25T10:52:32.453501+0000","last_active":"2024-04-25T10:53:31.152895+0000","last_peered":"2024-04-25T10:53:31.152895+0000","last_clean":"2024-04-25T10:53:31.152895+0000","last_became_active":"2024-04-25T10:52:32.453113+0000","last_became_peered":"2024-04-25T10:52:32.453113+0000","last_unstale":"2024-04-25T10:53:31.152895+0000","last_undegraded":"2024-04-25T10:53:31.152895+0000","last_fullsized":"2024-04-25T10:53:31.152895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:07.340709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148863+0000","last_change":"2024-04-25T10:52:31.697946+0000","last_active":"2024-04-25T10:53:31.148863+0000","last_peered":"2024-04-25T10:53:31.148863+0000","last_clean":"2024-04-25T10:53:31.148863+0000","last_became_active":"2024-04-25T10:52:31.697612+0000","last_became_peered":"2024-04-25T10:52:31.697612+0000","last_unstale":"2024-04-25T10:53:31.148863+0000","last_undegraded":"2024-04-25T10:53:31.148863+0000","last_fullsized":"2024-04-25T10:53:31.148863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.280776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150205+0000","last_change":"2024-04-25T10:52:32.473537+0000","last_active":"2024-04-25T10:53:31.150205+0000","last_peered":"2024-04-25T10:53:31.150205+0000","last_clean":"2024-04-25T10:53:31.150205+0000","last_became_active":"2024-04-25T10:52:32.473164+0000","last_became_peered":"2024-04-25T10:52:32.473164+0000","last_unstale":"2024-04-25T10:53:31.150205+0000","last_undegraded":"2024-04-25T10:53:31.150205+0000","last_fullsized":"2024-04-25T10:53:31.150205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.008031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244011+0000","last_change":"2024-04-25T10:52:31.694207+0000","last_active":"2024-04-25T10:53:31.244011+0000","last_peered":"2024-04-25T10:53:31.244011+0000","last_clean":"2024-04-25T10:53:31.244011+0000","last_became_active":"2024-04-25T10:52:31.693777+0000","last_became_peered":"2024-04-25T10:52:31.693777+0000","last_unstale":"2024-04-25T10:53:31.244011+0000","last_undegraded":"2024-04-25T10:53:31.244011+0000","last_fullsized":"2024-04-25T10:53:31.244011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:43.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146864+0000","last_change":"2024-04-25T10:52:32.475545+0000","last_active":"2024-04-25T10:53:31.146864+0000","last_peered":"2024-04-25T10:53:31.146864+0000","last_clean":"2024-04-25T10:53:31.146864+0000","last_became_active":"2024-04-25T10:52:32.475195+0000","last_became_peered":"2024-04-25T10:52:32.475195+0000","last_unstale":"2024-04-25T10:53:31.146864+0000","last_undegraded":"2024-04-25T10:53:31.146864+0000","last_fullsized":"2024-04-25T10:53:31.146864+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:49.900374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150024+0000","last_change":"2024-04-25T10:52:31.698184+0000","last_active":"2024-04-25T10:53:31.150024+0000","last_peered":"2024-04-25T10:53:31.150024+0000","last_clean":"2024-04-25T10:53:31.150024+0000","last_became_active":"2024-04-25T10:52:31.697816+0000","last_became_peered":"2024-04-25T10:52:31.697816+0000","last_unstale":"2024-04-25T10:53:31.150024+0000","last_undegraded":"2024-04-25T10:53:31.150024+0000","last_fullsized":"2024-04-25T10:53:31.150024+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:44.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300445+0000","last_change":"2024-04-25T10:52:32.472437+0000","last_active":"2024-04-25T10:53:31.300445+0000","last_peered":"2024-04-25T10:53:31.300445+0000","last_clean":"2024-04-25T10:53:31.300445+0000","last_became_active":"2024-04-25T10:52:32.471944+0000","last_became_peered":"2024-04-25T10:52:32.471944+0000","last_unstale":"2024-04-25T10:53:31.300445+0000","last_undegraded":"2024-04-25T10:53:31.300445+0000","last_fullsized":"2024-04-25T10:53:31.300445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:11.047158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149519+0000","last_change":"2024-04-25T10:52:31.696943+0000","last_active":"2024-04-25T10:53:31.149519+0000","last_peered":"2024-04-25T10:53:31.149519+0000","last_clean":"2024-04-25T10:53:31.149519+0000","last_became_active":"2024-04-25T10:52:31.696561+0000","last_became_peered":"2024-04-25T10:52:31.696561+0000","last_unstale":"2024-04-25T10:53:31.149519+0000","last_undegraded":"2024-04-25T10:53:31.149519+0000","last_fullsized":"2024-04-25T10:53:31.149519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:14.628800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146568+0000","last_change":"2024-04-25T10:52:32.461966+0000","last_active":"2024-04-25T10:53:31.146568+0000","last_peered":"2024-04-25T10:53:31.146568+0000","last_clean":"2024-04-25T10:53:31.146568+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:53:31.146568+0000","last_undegraded":"2024-04-25T10:53:31.146568+0000","last_fullsized":"2024-04-25T10:53:31.146568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'26","reported_seq":192,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:44.875184+0000","last_change":"2024-04-25T10:52:31.690938+0000","last_active":"2024-04-25T10:53:44.875184+0000","last_peered":"2024-04-25T10:53:44.875184+0000","last_clean":"2024-04-25T10:53:44.875184+0000","last_became_active":"2024-04-25T10:52:31.690511+0000","last_became_peered":"2024-04-25T10:52:31.690511+0000","last_unstale":"2024-04-25T10:53:44.875184+0000","last_undegraded":"2024-04-25T10:53:44.875184+0000","last_fullsized":"2024-04-25T10:53:44.875184+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:10.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":147,"num_read_kb":132,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147193+0000","last_change":"2024-04-25T10:52:32.474871+0000","last_active":"2024-04-25T10:53:31.147193+0000","last_peered":"2024-04-25T10:53:31.147193+0000","last_clean":"2024-04-25T10:53:31.147193+0000","last_became_active":"2024-04-25T10:52:32.474463+0000","last_became_peered":"2024-04-25T10:52:32.474463+0000","last_unstale":"2024-04-25T10:53:31.147193+0000","last_undegraded":"2024-04-25T10:53:31.147193+0000","last_fullsized":"2024-04-25T10:53:31.147193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:36.719230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'2","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242599+0000","last_change":"2024-04-25T10:52:31.691006+0000","last_active":"2024-04-25T10:53:31.242599+0000","last_peered":"2024-04-25T10:53:31.242599+0000","last_clean":"2024-04-25T10:53:31.242599+0000","last_became_active":"2024-04-25T10:52:31.690611+0000","last_became_peered":"2024-04-25T10:52:31.690611+0000","last_unstale":"2024-04-25T10:53:31.242599+0000","last_undegraded":"2024-04-25T10:53:31.242599+0000","last_fullsized":"2024-04-25T10:53:31.242599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:37.781160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145596+0000","last_change":"2024-04-25T10:52:32.463331+0000","last_active":"2024-04-25T10:53:31.145596+0000","last_peered":"2024-04-25T10:53:31.145596+0000","last_clean":"2024-04-25T10:53:31.145596+0000","last_became_active":"2024-04-25T10:52:32.462579+0000","last_became_peered":"2024-04-25T10:52:32.462579+0000","last_unstale":"2024-04-25T10:53:31.145596+0000","last_undegraded":"2024-04-25T10:53:31.145596+0000","last_fullsized":"2024-04-25T10:53:31.145596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148723+0000","last_change":"2024-04-25T10:52:31.704675+0000","last_active":"2024-04-25T10:53:31.148723+0000","last_peered":"2024-04-25T10:53:31.148723+0000","last_clean":"2024-04-25T10:53:31.148723+0000","last_became_active":"2024-04-25T10:52:31.704236+0000","last_became_peered":"2024-04-25T10:52:31.704236+0000","last_unstale":"2024-04-25T10:53:31.148723+0000","last_undegraded":"2024-04-25T10:53:31.148723+0000","last_fullsized":"2024-04-25T10:53:31.148723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.637006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150234+0000","last_change":"2024-04-25T10:52:32.462960+0000","last_active":"2024-04-25T10:53:31.150234+0000","last_peered":"2024-04-25T10:53:31.150234+0000","last_clean":"2024-04-25T10:53:31.150234+0000","last_became_active":"2024-04-25T10:52:32.461993+0000","last_became_peered":"2024-04-25T10:52:32.461993+0000","last_unstale":"2024-04-25T10:53:31.150234+0000","last_undegraded":"2024-04-25T10:53:31.150234+0000","last_fullsized":"2024-04-25T10:53:31.150234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:01.803209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243532+0000","last_change":"2024-04-25T10:52:31.698206+0000","last_active":"2024-04-25T10:53:31.243532+0000","last_peered":"2024-04-25T10:53:31.243532+0000","last_clean":"2024-04-25T10:53:31.243532+0000","last_became_active":"2024-04-25T10:52:31.697814+0000","last_became_peered":"2024-04-25T10:52:31.697814+0000","last_unstale":"2024-04-25T10:53:31.243532+0000","last_undegraded":"2024-04-25T10:53:31.243532+0000","last_fullsized":"2024-04-25T10:53:31.243532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:02.849229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149757+0000","last_change":"2024-04-25T10:52:32.473551+0000","last_active":"2024-04-25T10:53:31.149757+0000","last_peered":"2024-04-25T10:53:31.149757+0000","last_clean":"2024-04-25T10:53:31.149757+0000","last_became_active":"2024-04-25T10:52:32.473227+0000","last_became_peered":"2024-04-25T10:52:32.473227+0000","last_unstale":"2024-04-25T10:53:31.149757+0000","last_undegraded":"2024-04-25T10:53:31.149757+0000","last_fullsized":"2024-04-25T10:53:31.149757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:37.742066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'8","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146872+0000","last_change":"2024-04-25T10:52:31.700670+0000","last_active":"2024-04-25T10:53:31.146872+0000","last_peered":"2024-04-25T10:53:31.146872+0000","last_clean":"2024-04-25T10:53:31.146872+0000","last_became_active":"2024-04-25T10:52:31.700234+0000","last_became_peered":"2024-04-25T10:52:31.700234+0000","last_unstale":"2024-04-25T10:53:31.146872+0000","last_undegraded":"2024-04-25T10:53:31.146872+0000","last_fullsized":"2024-04-25T10:53:31.146872+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:21.795784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153193+0000","last_change":"2024-04-25T10:53:31.153193+0000","last_active":"2024-04-25T10:53:31.153193+0000","last_peered":"2024-04-25T10:53:31.153193+0000","last_clean":"2024-04-25T10:53:31.153193+0000","last_became_active":"2024-04-25T10:53:31.152723+0000","last_became_peered":"2024-04-25T10:53:31.152723+0000","last_unstale":"2024-04-25T10:53:31.153193+0000","last_undegraded":"2024-04-25T10:53:31.153193+0000","last_fullsized":"2024-04-25T10:53:31.153193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:42.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147445+0000","last_change":"2024-04-25T10:52:31.698507+0000","last_active":"2024-04-25T10:53:31.147445+0000","last_peered":"2024-04-25T10:53:31.147445+0000","last_clean":"2024-04-25T10:53:31.147445+0000","last_became_active":"2024-04-25T10:52:31.698148+0000","last_became_peered":"2024-04-25T10:52:31.698148+0000","last_unstale":"2024-04-25T10:53:31.147445+0000","last_undegraded":"2024-04-25T10:53:31.147445+0000","last_fullsized":"2024-04-25T10:53:31.147445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:48.993234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149622+0000","last_change":"2024-04-25T10:52:32.465874+0000","last_active":"2024-04-25T10:53:31.149622+0000","last_peered":"2024-04-25T10:53:31.149622+0000","last_clean":"2024-04-25T10:53:31.149622+0000","last_became_active":"2024-04-25T10:52:32.465511+0000","last_became_peered":"2024-04-25T10:52:32.465511+0000","last_unstale":"2024-04-25T10:53:31.149622+0000","last_undegraded":"2024-04-25T10:53:31.149622+0000","last_fullsized":"2024-04-25T10:53:31.149622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:15.274909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149935+0000","last_change":"2024-04-25T10:52:31.691074+0000","last_active":"2024-04-25T10:53:31.149935+0000","last_peered":"2024-04-25T10:53:31.149935+0000","last_clean":"2024-04-25T10:53:31.149935+0000","last_became_active":"2024-04-25T10:52:31.690402+0000","last_became_peered":"2024-04-25T10:52:31.690402+0000","last_unstale":"2024-04-25T10:53:31.149935+0000","last_undegraded":"2024-04-25T10:53:31.149935+0000","last_fullsized":"2024-04-25T10:53:31.149935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.529860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244120+0000","last_change":"2024-04-25T10:52:32.477968+0000","last_active":"2024-04-25T10:53:31.244120+0000","last_peered":"2024-04-25T10:53:31.244120+0000","last_clean":"2024-04-25T10:53:31.244120+0000","last_became_active":"2024-04-25T10:52:32.476395+0000","last_became_peered":"2024-04-25T10:52:32.476395+0000","last_unstale":"2024-04-25T10:53:31.244120+0000","last_undegraded":"2024-04-25T10:53:31.244120+0000","last_fullsized":"2024-04-25T10:53:31.244120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:13.128910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149922+0000","last_change":"2024-04-25T10:52:31.698934+0000","last_active":"2024-04-25T10:53:31.149922+0000","last_peered":"2024-04-25T10:53:31.149922+0000","last_clean":"2024-04-25T10:53:31.149922+0000","last_became_active":"2024-04-25T10:52:31.698565+0000","last_became_peered":"2024-04-25T10:52:31.698565+0000","last_unstale":"2024-04-25T10:53:31.149922+0000","last_undegraded":"2024-04-25T10:53:31.149922+0000","last_fullsized":"2024-04-25T10:53:31.149922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.360617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244087+0000","last_change":"2024-04-25T10:52:32.477538+0000","last_active":"2024-04-25T10:53:31.244087+0000","last_peered":"2024-04-25T10:53:31.244087+0000","last_clean":"2024-04-25T10:53:31.244087+0000","last_became_active":"2024-04-25T10:52:32.476088+0000","last_became_peered":"2024-04-25T10:52:32.476088+0000","last_unstale":"2024-04-25T10:53:31.244087+0000","last_undegraded":"2024-04-25T10:53:31.244087+0000","last_fullsized":"2024-04-25T10:53:31.244087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.987071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244447+0000","last_change":"2024-04-25T10:53:31.244447+0000","last_active":"2024-04-25T10:53:31.244447+0000","last_peered":"2024-04-25T10:53:31.244447+0000","last_clean":"2024-04-25T10:53:31.244447+0000","last_became_active":"2024-04-25T10:53:31.243944+0000","last_became_peered":"2024-04-25T10:53:31.243944+0000","last_unstale":"2024-04-25T10:53:31.244447+0000","last_undegraded":"2024-04-25T10:53:31.244447+0000","last_fullsized":"2024-04-25T10:53:31.244447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:46.344278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145467+0000","last_change":"2024-04-25T10:52:31.706700+0000","last_active":"2024-04-25T10:53:31.145467+0000","last_peered":"2024-04-25T10:53:31.145467+0000","last_clean":"2024-04-25T10:53:31.145467+0000","last_became_active":"2024-04-25T10:52:31.706392+0000","last_became_peered":"2024-04-25T10:52:31.706392+0000","last_unstale":"2024-04-25T10:53:31.145467+0000","last_undegraded":"2024-04-25T10:53:31.145467+0000","last_fullsized":"2024-04-25T10:53:31.145467+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:58.867351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153145+0000","last_change":"2024-04-25T10:52:32.462603+0000","last_active":"2024-04-25T10:53:31.153145+0000","last_peered":"2024-04-25T10:53:31.153145+0000","last_clean":"2024-04-25T10:53:31.153145+0000","last_became_active":"2024-04-25T10:52:32.462238+0000","last_became_peered":"2024-04-25T10:52:32.462238+0000","last_unstale":"2024-04-25T10:53:31.153145+0000","last_undegraded":"2024-04-25T10:53:31.153145+0000","last_fullsized":"2024-04-25T10:53:31.153145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.555683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243644+0000","last_change":"2024-04-25T10:52:31.694054+0000","last_active":"2024-04-25T10:53:31.243644+0000","last_peered":"2024-04-25T10:53:31.243644+0000","last_clean":"2024-04-25T10:53:31.243644+0000","last_became_active":"2024-04-25T10:52:31.693693+0000","last_became_peered":"2024-04-25T10:52:31.693693+0000","last_unstale":"2024-04-25T10:53:31.243644+0000","last_undegraded":"2024-04-25T10:53:31.243644+0000","last_fullsized":"2024-04-25T10:53:31.243644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:22.238131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149828+0000","last_change":"2024-04-25T10:51:22.730925+0000","last_active":"2024-04-25T10:53:31.149828+0000","last_peered":"2024-04-25T10:53:31.149828+0000","last_clean":"2024-04-25T10:53:31.149828+0000","last_became_active":"2024-04-25T10:51:22.725168+0000","last_became_peered":"2024-04-25T10:51:22.725168+0000","last_unstale":"2024-04-25T10:53:31.149828+0000","last_undegraded":"2024-04-25T10:53:31.149828+0000","last_fullsized":"2024-04-25T10:53:31.149828+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_clean_scrub_stamp":"2024-04-25T10:47:56.758246+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:02.416333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145464+0000","last_change":"2024-04-25T10:52:32.461608+0000","last_active":"2024-04-25T10:53:31.145464+0000","last_peered":"2024-04-25T10:53:31.145464+0000","last_clean":"2024-04-25T10:53:31.145464+0000","last_became_active":"2024-04-25T10:52:32.461210+0000","last_became_peered":"2024-04-25T10:52:32.461210+0000","last_unstale":"2024-04-25T10:53:31.145464+0000","last_undegraded":"2024-04-25T10:53:31.145464+0000","last_fullsized":"2024-04-25T10:53:31.145464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:14.897053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150087+0000","last_change":"2024-04-25T10:52:31.701003+0000","last_active":"2024-04-25T10:53:31.150087+0000","last_peered":"2024-04-25T10:53:31.150087+0000","last_clean":"2024-04-25T10:53:31.150087+0000","last_became_active":"2024-04-25T10:52:31.699623+0000","last_became_peered":"2024-04-25T10:52:31.699623+0000","last_unstale":"2024-04-25T10:53:31.150087+0000","last_undegraded":"2024-04-25T10:53:31.150087+0000","last_fullsized":"2024-04-25T10:53:31.150087+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:45.175760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147395+0000","last_change":"2024-04-25T10:52:31.702595+0000","last_active":"2024-04-25T10:53:31.147395+0000","last_peered":"2024-04-25T10:53:31.147395+0000","last_clean":"2024-04-25T10:53:31.147395+0000","last_became_active":"2024-04-25T10:52:31.702124+0000","last_became_peered":"2024-04-25T10:52:31.702124+0000","last_unstale":"2024-04-25T10:53:31.147395+0000","last_undegraded":"2024-04-25T10:53:31.147395+0000","last_fullsized":"2024-04-25T10:53:31.147395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:54.351859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145848+0000","last_change":"2024-04-25T10:52:32.467987+0000","last_active":"2024-04-25T10:53:31.145848+0000","last_peered":"2024-04-25T10:53:31.145848+0000","last_clean":"2024-04-25T10:53:31.145848+0000","last_became_active":"2024-04-25T10:52:32.467613+0000","last_became_peered":"2024-04-25T10:52:32.467613+0000","last_unstale":"2024-04-25T10:53:31.145848+0000","last_undegraded":"2024-04-25T10:53:31.145848+0000","last_fullsized":"2024-04-25T10:53:31.145848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:00.112102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'6","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150152+0000","last_change":"2024-04-25T10:52:31.699868+0000","last_active":"2024-04-25T10:53:31.150152+0000","last_peered":"2024-04-25T10:53:31.150152+0000","last_clean":"2024-04-25T10:53:31.150152+0000","last_became_active":"2024-04-25T10:52:31.699470+0000","last_became_peered":"2024-04-25T10:52:31.699470+0000","last_unstale":"2024-04-25T10:53:31.150152+0000","last_undegraded":"2024-04-25T10:53:31.150152+0000","last_fullsized":"2024-04-25T10:53:31.150152+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.161821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154082+0000","last_change":"2024-04-25T10:52:32.459082+0000","last_active":"2024-04-25T10:53:31.154082+0000","last_peered":"2024-04-25T10:53:31.154082+0000","last_clean":"2024-04-25T10:53:31.154082+0000","last_became_active":"2024-04-25T10:52:32.458190+0000","last_became_peered":"2024-04-25T10:52:32.458190+0000","last_unstale":"2024-04-25T10:53:31.154082+0000","last_undegraded":"2024-04-25T10:53:31.154082+0000","last_fullsized":"2024-04-25T10:53:31.154082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.280762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242887+0000","last_change":"2024-04-25T10:52:31.700636+0000","last_active":"2024-04-25T10:53:31.242887+0000","last_peered":"2024-04-25T10:53:31.242887+0000","last_clean":"2024-04-25T10:53:31.242887+0000","last_became_active":"2024-04-25T10:52:31.700268+0000","last_became_peered":"2024-04-25T10:52:31.700268+0000","last_unstale":"2024-04-25T10:53:31.242887+0000","last_undegraded":"2024-04-25T10:53:31.242887+0000","last_fullsized":"2024-04-25T10:53:31.242887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.439491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300393+0000","last_change":"2024-04-25T10:52:32.467406+0000","last_active":"2024-04-25T10:53:31.300393+0000","last_peered":"2024-04-25T10:53:31.300393+0000","last_clean":"2024-04-25T10:53:31.300393+0000","last_became_active":"2024-04-25T10:52:32.467053+0000","last_became_peered":"2024-04-25T10:52:32.467053+0000","last_unstale":"2024-04-25T10:53:31.300393+0000","last_undegraded":"2024-04-25T10:53:31.300393+0000","last_fullsized":"2024-04-25T10:53:31.300393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.301411+0000","last_change":"2024-04-25T10:53:31.301411+0000","last_active":"2024-04-25T10:53:31.301411+0000","last_peered":"2024-04-25T10:53:31.301411+0000","last_clean":"2024-04-25T10:53:31.301411+0000","last_became_active":"2024-04-25T10:53:31.301022+0000","last_became_peered":"2024-04-25T10:53:31.301022+0000","last_unstale":"2024-04-25T10:53:31.301411+0000","last_undegraded":"2024-04-25T10:53:31.301411+0000","last_fullsized":"2024-04-25T10:53:31.301411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:28.171444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146876+0000","last_change":"2024-04-25T10:52:32.472709+0000","last_active":"2024-04-25T10:53:31.146876+0000","last_peered":"2024-04-25T10:53:31.146876+0000","last_clean":"2024-04-25T10:53:31.146876+0000","last_became_active":"2024-04-25T10:52:32.472331+0000","last_became_peered":"2024-04-25T10:52:32.472331+0000","last_unstale":"2024-04-25T10:53:31.146876+0000","last_undegraded":"2024-04-25T10:53:31.146876+0000","last_fullsized":"2024-04-25T10:53:31.146876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.047682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153117+0000","last_change":"2024-04-25T10:52:31.697360+0000","last_active":"2024-04-25T10:53:31.153117+0000","last_peered":"2024-04-25T10:53:31.153117+0000","last_clean":"2024-04-25T10:53:31.153117+0000","last_became_active":"2024-04-25T10:52:31.696991+0000","last_became_peered":"2024-04-25T10:52:31.696991+0000","last_unstale":"2024-04-25T10:53:31.153117+0000","last_undegraded":"2024-04-25T10:53:31.153117+0000","last_fullsized":"2024-04-25T10:53:31.153117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:38.395372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150674+0000","last_change":"2024-04-25T10:52:32.472505+0000","last_active":"2024-04-25T10:53:31.150674+0000","last_peered":"2024-04-25T10:53:31.150674+0000","last_clean":"2024-04-25T10:53:31.150674+0000","last_became_active":"2024-04-25T10:52:32.472113+0000","last_became_peered":"2024-04-25T10:52:32.472113+0000","last_unstale":"2024-04-25T10:53:31.150674+0000","last_undegraded":"2024-04-25T10:53:31.150674+0000","last_fullsized":"2024-04-25T10:53:31.150674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:20.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242458+0000","last_change":"2024-04-25T10:52:31.696488+0000","last_active":"2024-04-25T10:53:31.242458+0000","last_peered":"2024-04-25T10:53:31.242458+0000","last_clean":"2024-04-25T10:53:31.242458+0000","last_became_active":"2024-04-25T10:52:31.696062+0000","last_became_peered":"2024-04-25T10:52:31.696062+0000","last_unstale":"2024-04-25T10:53:31.242458+0000","last_undegraded":"2024-04-25T10:53:31.242458+0000","last_fullsized":"2024-04-25T10:53:31.242458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:24.329938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153458+0000","last_change":"2024-04-25T10:52:32.474555+0000","last_active":"2024-04-25T10:53:31.153458+0000","last_peered":"2024-04-25T10:53:31.153458+0000","last_clean":"2024-04-25T10:53:31.153458+0000","last_became_active":"2024-04-25T10:52:32.474001+0000","last_became_peered":"2024-04-25T10:52:32.474001+0000","last_unstale":"2024-04-25T10:53:31.153458+0000","last_undegraded":"2024-04-25T10:53:31.153458+0000","last_fullsized":"2024-04-25T10:53:31.153458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.775966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'2","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146979+0000","last_change":"2024-04-25T10:52:31.702251+0000","last_active":"2024-04-25T10:53:31.146979+0000","last_peered":"2024-04-25T10:53:31.146979+0000","last_clean":"2024-04-25T10:53:31.146979+0000","last_became_active":"2024-04-25T10:52:31.701763+0000","last_became_peered":"2024-04-25T10:52:31.701763+0000","last_unstale":"2024-04-25T10:53:31.146979+0000","last_undegraded":"2024-04-25T10:53:31.146979+0000","last_fullsized":"2024-04-25T10:53:31.146979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150054+0000","last_change":"2024-04-25T10:52:32.465494+0000","last_active":"2024-04-25T10:53:31.150054+0000","last_peered":"2024-04-25T10:53:31.150054+0000","last_clean":"2024-04-25T10:53:31.150054+0000","last_became_active":"2024-04-25T10:52:32.463855+0000","last_became_peered":"2024-04-25T10:52:32.463855+0000","last_unstale":"2024-04-25T10:53:31.150054+0000","last_undegraded":"2024-04-25T10:53:31.150054+0000","last_fullsized":"2024-04-25T10:53:31.150054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:40.970580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242816+0000","last_change":"2024-04-25T10:52:31.700749+0000","last_active":"2024-04-25T10:53:31.242816+0000","last_peered":"2024-04-25T10:53:31.242816+0000","last_clean":"2024-04-25T10:53:31.242816+0000","last_became_active":"2024-04-25T10:52:31.699109+0000","last_became_peered":"2024-04-25T10:52:31.699109+0000","last_unstale":"2024-04-25T10:53:31.242816+0000","last_undegraded":"2024-04-25T10:53:31.242816+0000","last_fullsized":"2024-04-25T10:53:31.242816+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:48.405333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150398+0000","last_change":"2024-04-25T10:52:32.468292+0000","last_active":"2024-04-25T10:53:31.150398+0000","last_peered":"2024-04-25T10:53:31.150398+0000","last_clean":"2024-04-25T10:53:31.150398+0000","last_became_active":"2024-04-25T10:52:32.467934+0000","last_became_peered":"2024-04-25T10:52:32.467934+0000","last_unstale":"2024-04-25T10:53:31.150398+0000","last_undegraded":"2024-04-25T10:53:31.150398+0000","last_fullsized":"2024-04-25T10:53:31.150398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.366735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149325+0000","last_change":"2024-04-25T10:52:31.705025+0000","last_active":"2024-04-25T10:53:31.149325+0000","last_peered":"2024-04-25T10:53:31.149325+0000","last_clean":"2024-04-25T10:53:31.149325+0000","last_became_active":"2024-04-25T10:52:31.704532+0000","last_became_peered":"2024-04-25T10:52:31.704532+0000","last_unstale":"2024-04-25T10:53:31.149325+0000","last_undegraded":"2024-04-25T10:53:31.149325+0000","last_fullsized":"2024-04-25T10:53:31.149325+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.371778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.158758+0000","last_change":"2024-04-25T10:53:31.158758+0000","last_active":"2024-04-25T10:53:31.158758+0000","last_peered":"2024-04-25T10:53:31.158758+0000","last_clean":"2024-04-25T10:53:31.158758+0000","last_became_active":"2024-04-25T10:53:31.152575+0000","last_became_peered":"2024-04-25T10:53:31.152575+0000","last_unstale":"2024-04-25T10:53:31.158758+0000","last_undegraded":"2024-04-25T10:53:31.158758+0000","last_fullsized":"2024-04-25T10:53:31.158758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145781+0000","last_change":"2024-04-25T10:52:31.705770+0000","last_active":"2024-04-25T10:53:31.145781+0000","last_peered":"2024-04-25T10:53:31.145781+0000","last_clean":"2024-04-25T10:53:31.145781+0000","last_became_active":"2024-04-25T10:52:31.705391+0000","last_became_peered":"2024-04-25T10:52:31.705391+0000","last_unstale":"2024-04-25T10:53:31.145781+0000","last_undegraded":"2024-04-25T10:53:31.145781+0000","last_fullsized":"2024-04-25T10:53:31.145781+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:32.831435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150206+0000","last_change":"2024-04-25T10:52:32.465550+0000","last_active":"2024-04-25T10:53:31.150206+0000","last_peered":"2024-04-25T10:53:31.150206+0000","last_clean":"2024-04-25T10:53:31.150206+0000","last_became_active":"2024-04-25T10:52:32.465034+0000","last_became_peered":"2024-04-25T10:52:32.465034+0000","last_unstale":"2024-04-25T10:53:31.150206+0000","last_undegraded":"2024-04-25T10:53:31.150206+0000","last_fullsized":"2024-04-25T10:53:31.150206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.810469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150713+0000","last_change":"2024-04-25T10:52:32.470812+0000","last_active":"2024-04-25T10:53:31.150713+0000","last_peered":"2024-04-25T10:53:31.150713+0000","last_clean":"2024-04-25T10:53:31.150713+0000","last_became_active":"2024-04-25T10:52:32.469620+0000","last_became_peered":"2024-04-25T10:52:32.469620+0000","last_unstale":"2024-04-25T10:53:31.150713+0000","last_undegraded":"2024-04-25T10:53:31.150713+0000","last_fullsized":"2024-04-25T10:53:31.150713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242788+0000","last_change":"2024-04-25T10:52:32.464762+0000","last_active":"2024-04-25T10:53:31.242788+0000","last_peered":"2024-04-25T10:53:31.242788+0000","last_clean":"2024-04-25T10:53:31.242788+0000","last_became_active":"2024-04-25T10:52:32.464316+0000","last_became_peered":"2024-04-25T10:52:32.464316+0000","last_unstale":"2024-04-25T10:53:31.242788+0000","last_undegraded":"2024-04-25T10:53:31.242788+0000","last_fullsized":"2024-04-25T10:53:31.242788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:55.526572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242433+0000","last_change":"2024-04-25T10:52:32.466429+0000","last_active":"2024-04-25T10:53:31.242433+0000","last_peered":"2024-04-25T10:53:31.242433+0000","last_clean":"2024-04-25T10:53:31.242433+0000","last_became_active":"2024-04-25T10:52:32.465132+0000","last_became_peered":"2024-04-25T10:52:32.465132+0000","last_unstale":"2024-04-25T10:53:31.242433+0000","last_undegraded":"2024-04-25T10:53:31.242433+0000","last_fullsized":"2024-04-25T10:53:31.242433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:14.274441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300373+0000","last_change":"2024-04-25T10:52:32.464666+0000","last_active":"2024-04-25T10:53:31.300373+0000","last_peered":"2024-04-25T10:53:31.300373+0000","last_clean":"2024-04-25T10:53:31.300373+0000","last_became_active":"2024-04-25T10:52:32.464226+0000","last_became_peered":"2024-04-25T10:52:32.464226+0000","last_unstale":"2024-04-25T10:53:31.300373+0000","last_undegraded":"2024-04-25T10:53:31.300373+0000","last_fullsized":"2024-04-25T10:53:31.300373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:39.388542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145725+0000","last_change":"2024-04-25T10:52:32.465199+0000","last_active":"2024-04-25T10:53:31.145725+0000","last_peered":"2024-04-25T10:53:31.145725+0000","last_clean":"2024-04-25T10:53:31.145725+0000","last_became_active":"2024-04-25T10:52:32.464285+0000","last_became_peered":"2024-04-25T10:52:32.464285+0000","last_unstale":"2024-04-25T10:53:31.145725+0000","last_undegraded":"2024-04-25T10:53:31.145725+0000","last_fullsized":"2024-04-25T10:53:31.145725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.010994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149373+0000","last_change":"2024-04-25T10:52:32.469140+0000","last_active":"2024-04-25T10:53:31.149373+0000","last_peered":"2024-04-25T10:53:31.149373+0000","last_clean":"2024-04-25T10:53:31.149373+0000","last_became_active":"2024-04-25T10:52:32.468344+0000","last_became_peered":"2024-04-25T10:52:32.468344+0000","last_unstale":"2024-04-25T10:53:31.149373+0000","last_undegraded":"2024-04-25T10:53:31.149373+0000","last_fullsized":"2024-04-25T10:53:31.149373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:07.228059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243075+0000","last_change":"2024-04-25T10:52:32.469326+0000","last_active":"2024-04-25T10:53:31.243075+0000","last_peered":"2024-04-25T10:53:31.243075+0000","last_clean":"2024-04-25T10:53:31.243075+0000","last_became_active":"2024-04-25T10:52:32.468905+0000","last_became_peered":"2024-04-25T10:52:32.468905+0000","last_unstale":"2024-04-25T10:53:31.243075+0000","last_undegraded":"2024-04-25T10:53:31.243075+0000","last_fullsized":"2024-04-25T10:53:31.243075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:31.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154039+0000","last_change":"2024-04-25T10:52:32.459391+0000","last_active":"2024-04-25T10:53:31.154039+0000","last_peered":"2024-04-25T10:53:31.154039+0000","last_clean":"2024-04-25T10:53:31.154039+0000","last_became_active":"2024-04-25T10:52:32.458702+0000","last_became_peered":"2024-04-25T10:52:32.458702+0000","last_unstale":"2024-04-25T10:53:31.154039+0000","last_undegraded":"2024-04-25T10:53:31.154039+0000","last_fullsized":"2024-04-25T10:53:31.154039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.475714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148497+0000","last_change":"2024-04-25T10:52:32.467395+0000","last_active":"2024-04-25T10:53:31.148497+0000","last_peered":"2024-04-25T10:53:31.148497+0000","last_clean":"2024-04-25T10:53:31.148497+0000","last_became_active":"2024-04-25T10:52:32.466976+0000","last_became_peered":"2024-04-25T10:52:32.466976+0000","last_unstale":"2024-04-25T10:53:31.148497+0000","last_undegraded":"2024-04-25T10:53:31.148497+0000","last_fullsized":"2024-04-25T10:53:31.148497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:22.011064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152631+0000","last_change":"2024-04-25T10:52:32.458874+0000","last_active":"2024-04-25T10:53:31.152631+0000","last_peered":"2024-04-25T10:53:31.152631+0000","last_clean":"2024-04-25T10:53:31.152631+0000","last_became_active":"2024-04-25T10:52:32.458517+0000","last_became_peered":"2024-04-25T10:52:32.458517+0000","last_unstale":"2024-04-25T10:53:31.152631+0000","last_undegraded":"2024-04-25T10:53:31.152631+0000","last_fullsized":"2024-04-25T10:53:31.152631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:12.634521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146425+0000","last_change":"2024-04-25T10:52:32.473068+0000","last_active":"2024-04-25T10:53:31.146425+0000","last_peered":"2024-04-25T10:53:31.146425+0000","last_clean":"2024-04-25T10:53:31.146425+0000","last_became_active":"2024-04-25T10:52:32.472713+0000","last_became_peered":"2024-04-25T10:52:32.472713+0000","last_unstale":"2024-04-25T10:53:31.146425+0000","last_undegraded":"2024-04-25T10:53:31.146425+0000","last_fullsized":"2024-04-25T10:53:31.146425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:43.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145779+0000","last_change":"2024-04-25T10:52:32.468492+0000","last_active":"2024-04-25T10:53:31.145779+0000","last_peered":"2024-04-25T10:53:31.145779+0000","last_clean":"2024-04-25T10:53:31.145779+0000","last_became_active":"2024-04-25T10:52:32.468155+0000","last_became_peered":"2024-04-25T10:52:32.468155+0000","last_unstale":"2024-04-25T10:53:31.145779+0000","last_undegraded":"2024-04-25T10:53:31.145779+0000","last_fullsized":"2024-04-25T10:53:31.145779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.934230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149774+0000","last_change":"2024-04-25T10:52:32.463057+0000","last_active":"2024-04-25T10:53:31.149774+0000","last_peered":"2024-04-25T10:53:31.149774+0000","last_clean":"2024-04-25T10:53:31.149774+0000","last_became_active":"2024-04-25T10:52:32.460983+0000","last_became_peered":"2024-04-25T10:52:32.460983+0000","last_unstale":"2024-04-25T10:53:31.149774+0000","last_undegraded":"2024-04-25T10:53:31.149774+0000","last_fullsized":"2024-04-25T10:53:31.149774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":153,"num_read_kb":138,"num_write":75,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":843776,"data_stored":411149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1523712,"data_stored":1062874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":411848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":411753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1445888,"data_stored":1002216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593781,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":847872,"data_stored":411266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":472195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":411239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":472229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":411370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":411171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1454080,"data_stored":1001816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:53:47.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T10:53:48.019 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:48.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:48 smithi019 ceph-mon[29209]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:53:48.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:48 smithi169 ceph-mon[33049]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:53:48.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:48 smithi112 ceph-mon[32799]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:53:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:49 smithi169 ceph-mon[33049]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:49.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:49 smithi019 ceph-mon[29209]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:49 smithi112 ceph-mon[32799]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 -- 172.21.15.19:0/1124891479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 msgr2=0x7f0168102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1124891479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f0168102a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0158009990 tx=0x7f0158009ca0 comp rx=0 tx=0).stop 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 -- 172.21.15.19:0/1124891479 shutdown_connections 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1124891479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0168102fa0 0x7f0168111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1124891479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01681091c0 0x7f01681095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1124891479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f0168102a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 -- 172.21.15.19:0/1124891479 >> 172.21.15.19:0/1124891479 conn(0x7f01680fe400 msgr2=0x7f0168100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 -- 172.21.15.19:0/1124891479 shutdown_connections 2024-04-25T10:53:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.771+0000 7f016d1fd700 1 -- 172.21.15.19:0/1124891479 wait complete. 2024-04-25T10:53:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.772+0000 7f016d1fd700 1 Processor -- start 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016d1fd700 1 -- start start 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016d1fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f016819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016d1fd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0168102fa0 0x7f016819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016d1fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01681091c0 0x7f016819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016d1fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01681144b0 con 0x7f0168102650 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016d1fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0168114330 con 0x7f01681091c0 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016d1fd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0168114630 con 0x7f0168102fa0 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f016819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f016819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59134/0 (socket says 172.21.15.19:59134) 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 -- 172.21.15.19:0/1864422188 learned_addr learned my addr 172.21.15.19:0/1864422188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016759e700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01681091c0 0x7f016819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f016659c700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0168102fa0 0x7f016819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 -- 172.21.15.19:0/1864422188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0168102fa0 msgr2=0x7f016819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0168102fa0 0x7f016819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 -- 172.21.15.19:0/1864422188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01681091c0 msgr2=0x7f016819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01681091c0 0x7f016819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.773+0000 7f0166d9d700 1 -- 172.21.15.19:0/1864422188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016819fc10 con 0x7f0168102650 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.774+0000 7f0166d9d700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f016819ae40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c970 tx=0x7f015c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.774+0000 7f014ffff700 1 -- 172.21.15.19:0/1864422188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c007b00 con 0x7f0168102650 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.774+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016819ff00 con 0x7f0168102650 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.774+0000 7f014ffff700 1 -- 172.21.15.19:0/1864422188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c007ca0 con 0x7f0168102650 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.774+0000 7f014ffff700 1 -- 172.21.15.19:0/1864422188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c018660 con 0x7f0168102650 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.774+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a78d0 con 0x7f0168102650 2024-04-25T10:53:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.775+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0168051190 con 0x7f0168102650 2024-04-25T10:53:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.777+0000 7f014ffff700 1 -- 172.21.15.19:0/1864422188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f015c00f450 con 0x7f0168102650 2024-04-25T10:53:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.777+0000 7f014ffff700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01500afc10 0x7f01500b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.777+0000 7f016659c700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01500afc10 0x7f01500b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.777+0000 7f014ffff700 1 -- 172.21.15.19:0/1864422188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f015c0dee20 con 0x7f0168102650 2024-04-25T10:53:49.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.778+0000 7f016659c700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01500afc10 0x7f01500b2120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0158006010 tx=0x7f015801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.781+0000 7f014ffff700 1 -- 172.21.15.19:0/1864422188 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f015c052ff0 con 0x7f0168102650 2024-04-25T10:53:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.984+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f016819c150 con 0x7f01500afc10 2024-04-25T10:53:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.992+0000 7f014ffff700 1 -- 172.21.15.19:0/1864422188 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258626 (secure 0 0 0) 0x7f016819c150 con 0x7f01500afc10 2024-04-25T10:53:49.997 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01500afc10 msgr2=0x7f01500b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01500afc10 0x7f01500b2120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0158006010 tx=0x7f015801a040 comp rx=0 tx=0).stop 2024-04-25T10:53:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 msgr2=0x7f016819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f016819ae40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c970 tx=0x7f015c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:53:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 shutdown_connections 2024-04-25T10:53:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f01500afc10 0x7f01500b2120 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01681091c0 0x7f016819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0168102650 0x7f016819ae40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 --2- 172.21.15.19:0/1864422188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0168102fa0 0x7f016819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 >> 172.21.15.19:0/1864422188 conn(0x7f01680fe400 msgr2=0x7f01681070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 shutdown_connections 2024-04-25T10:53:50.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:49.995+0000 7f016d1fd700 1 -- 172.21.15.19:0/1864422188 wait complete. 2024-04-25T10:53:50.003 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:53:50.814 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-25T10:53:48.012969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":234,"num_read_kb":207,"num_write":200,"num_write_kb":2235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365304,"kb_used_data":11916,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576520,"statfs":{"total":1151940427776,"available":1151566356480,"internally_reserved":0,"allocated":12201984,"data_stored":6891126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003499"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150151+0000","last_change":"2024-04-25T10:52:32.474738+0000","last_active":"2024-04-25T10:53:31.150151+0000","last_peered":"2024-04-25T10:53:31.150151+0000","last_clean":"2024-04-25T10:53:31.150151+0000","last_became_active":"2024-04-25T10:52:32.473812+0000","last_became_peered":"2024-04-25T10:52:32.473812+0000","last_unstale":"2024-04-25T10:53:31.150151+0000","last_undegraded":"2024-04-25T10:53:31.150151+0000","last_fullsized":"2024-04-25T10:53:31.150151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:25.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300204+0000","last_change":"2024-04-25T10:52:32.475491+0000","last_active":"2024-04-25T10:53:31.300204+0000","last_peered":"2024-04-25T10:53:31.300204+0000","last_clean":"2024-04-25T10:53:31.300204+0000","last_became_active":"2024-04-25T10:52:32.474164+0000","last_became_peered":"2024-04-25T10:52:32.474164+0000","last_unstale":"2024-04-25T10:53:31.300204+0000","last_undegraded":"2024-04-25T10:53:31.300204+0000","last_fullsized":"2024-04-25T10:53:31.300204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:58.838292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300309+0000","last_change":"2024-04-25T10:52:32.471349+0000","last_active":"2024-04-25T10:53:31.300309+0000","last_peered":"2024-04-25T10:53:31.300309+0000","last_clean":"2024-04-25T10:53:31.300309+0000","last_became_active":"2024-04-25T10:52:32.470972+0000","last_became_peered":"2024-04-25T10:52:32.470972+0000","last_unstale":"2024-04-25T10:53:31.300309+0000","last_undegraded":"2024-04-25T10:53:31.300309+0000","last_fullsized":"2024-04-25T10:53:31.300309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:28.634537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243291+0000","last_change":"2024-04-25T10:52:32.470212+0000","last_active":"2024-04-25T10:53:31.243291+0000","last_peered":"2024-04-25T10:53:31.243291+0000","last_clean":"2024-04-25T10:53:31.243291+0000","last_became_active":"2024-04-25T10:52:32.469642+0000","last_became_peered":"2024-04-25T10:52:32.469642+0000","last_unstale":"2024-04-25T10:53:31.243291+0000","last_undegraded":"2024-04-25T10:53:31.243291+0000","last_fullsized":"2024-04-25T10:53:31.243291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:08.231679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147067+0000","last_change":"2024-04-25T10:52:32.456568+0000","last_active":"2024-04-25T10:53:31.147067+0000","last_peered":"2024-04-25T10:53:31.147067+0000","last_clean":"2024-04-25T10:53:31.147067+0000","last_became_active":"2024-04-25T10:52:32.456051+0000","last_became_peered":"2024-04-25T10:52:32.456051+0000","last_unstale":"2024-04-25T10:53:31.147067+0000","last_undegraded":"2024-04-25T10:53:31.147067+0000","last_fullsized":"2024-04-25T10:53:31.147067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243529+0000","last_change":"2024-04-25T10:52:32.479781+0000","last_active":"2024-04-25T10:53:31.243529+0000","last_peered":"2024-04-25T10:53:31.243529+0000","last_clean":"2024-04-25T10:53:31.243529+0000","last_became_active":"2024-04-25T10:52:32.479213+0000","last_became_peered":"2024-04-25T10:52:32.479213+0000","last_unstale":"2024-04-25T10:53:31.243529+0000","last_undegraded":"2024-04-25T10:53:31.243529+0000","last_fullsized":"2024-04-25T10:53:31.243529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:16.239969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149836+0000","last_change":"2024-04-25T10:52:32.471430+0000","last_active":"2024-04-25T10:53:31.149836+0000","last_peered":"2024-04-25T10:53:31.149836+0000","last_clean":"2024-04-25T10:53:31.149836+0000","last_became_active":"2024-04-25T10:52:32.470518+0000","last_became_peered":"2024-04-25T10:52:32.470518+0000","last_unstale":"2024-04-25T10:53:31.149836+0000","last_undegraded":"2024-04-25T10:53:31.149836+0000","last_fullsized":"2024-04-25T10:53:31.149836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:12.846174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152841+0000","last_change":"2024-04-25T10:52:32.472545+0000","last_active":"2024-04-25T10:53:31.152841+0000","last_peered":"2024-04-25T10:53:31.152841+0000","last_clean":"2024-04-25T10:53:31.152841+0000","last_became_active":"2024-04-25T10:52:32.472151+0000","last_became_peered":"2024-04-25T10:52:32.472151+0000","last_unstale":"2024-04-25T10:53:31.152841+0000","last_undegraded":"2024-04-25T10:53:31.152841+0000","last_fullsized":"2024-04-25T10:53:31.152841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:36.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146931+0000","last_change":"2024-04-25T10:52:32.466223+0000","last_active":"2024-04-25T10:53:31.146931+0000","last_peered":"2024-04-25T10:53:31.146931+0000","last_clean":"2024-04-25T10:53:31.146931+0000","last_became_active":"2024-04-25T10:52:32.465498+0000","last_became_peered":"2024-04-25T10:52:32.465498+0000","last_unstale":"2024-04-25T10:53:31.146931+0000","last_undegraded":"2024-04-25T10:53:31.146931+0000","last_fullsized":"2024-04-25T10:53:31.146931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:25.924450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244273+0000","last_change":"2024-04-25T10:52:32.470601+0000","last_active":"2024-04-25T10:53:31.244273+0000","last_peered":"2024-04-25T10:53:31.244273+0000","last_clean":"2024-04-25T10:53:31.244273+0000","last_became_active":"2024-04-25T10:52:32.468984+0000","last_became_peered":"2024-04-25T10:52:32.468984+0000","last_unstale":"2024-04-25T10:53:31.244273+0000","last_undegraded":"2024-04-25T10:53:31.244273+0000","last_fullsized":"2024-04-25T10:53:31.244273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150337+0000","last_change":"2024-04-25T10:52:32.463907+0000","last_active":"2024-04-25T10:53:31.150337+0000","last_peered":"2024-04-25T10:53:31.150337+0000","last_clean":"2024-04-25T10:53:31.150337+0000","last_became_active":"2024-04-25T10:52:32.463536+0000","last_became_peered":"2024-04-25T10:52:32.463536+0000","last_unstale":"2024-04-25T10:53:31.150337+0000","last_undegraded":"2024-04-25T10:53:31.150337+0000","last_fullsized":"2024-04-25T10:53:31.150337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.503805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243169+0000","last_change":"2024-04-25T10:52:32.473646+0000","last_active":"2024-04-25T10:53:31.243169+0000","last_peered":"2024-04-25T10:53:31.243169+0000","last_clean":"2024-04-25T10:53:31.243169+0000","last_became_active":"2024-04-25T10:52:32.473310+0000","last_became_peered":"2024-04-25T10:52:32.473310+0000","last_unstale":"2024-04-25T10:53:31.243169+0000","last_undegraded":"2024-04-25T10:53:31.243169+0000","last_fullsized":"2024-04-25T10:53:31.243169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.691293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244988+0000","last_change":"2024-04-25T10:53:31.244988+0000","last_active":"2024-04-25T10:53:31.244988+0000","last_peered":"2024-04-25T10:53:31.244988+0000","last_clean":"2024-04-25T10:53:31.244988+0000","last_became_active":"2024-04-25T10:53:31.244535+0000","last_became_peered":"2024-04-25T10:53:31.244535+0000","last_unstale":"2024-04-25T10:53:31.244988+0000","last_undegraded":"2024-04-25T10:53:31.244988+0000","last_fullsized":"2024-04-25T10:53:31.244988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:14.205691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300514+0000","last_change":"2024-04-25T10:52:32.476104+0000","last_active":"2024-04-25T10:53:31.300514+0000","last_peered":"2024-04-25T10:53:31.300514+0000","last_clean":"2024-04-25T10:53:31.300514+0000","last_became_active":"2024-04-25T10:52:32.475342+0000","last_became_peered":"2024-04-25T10:52:32.475342+0000","last_unstale":"2024-04-25T10:53:31.300514+0000","last_undegraded":"2024-04-25T10:53:31.300514+0000","last_fullsized":"2024-04-25T10:53:31.300514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:37.087793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300100+0000","last_change":"2024-04-25T10:52:32.470263+0000","last_active":"2024-04-25T10:53:31.300100+0000","last_peered":"2024-04-25T10:53:31.300100+0000","last_clean":"2024-04-25T10:53:31.300100+0000","last_became_active":"2024-04-25T10:52:32.469873+0000","last_became_peered":"2024-04-25T10:52:32.469873+0000","last_unstale":"2024-04-25T10:53:31.300100+0000","last_undegraded":"2024-04-25T10:53:31.300100+0000","last_fullsized":"2024-04-25T10:53:31.300100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.349598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146088+0000","last_change":"2024-04-25T10:52:32.471851+0000","last_active":"2024-04-25T10:53:31.146088+0000","last_peered":"2024-04-25T10:53:31.146088+0000","last_clean":"2024-04-25T10:53:31.146088+0000","last_became_active":"2024-04-25T10:52:32.471477+0000","last_became_peered":"2024-04-25T10:52:32.471477+0000","last_unstale":"2024-04-25T10:53:31.146088+0000","last_undegraded":"2024-04-25T10:53:31.146088+0000","last_fullsized":"2024-04-25T10:53:31.146088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:59.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150020+0000","last_change":"2024-04-25T10:52:32.472530+0000","last_active":"2024-04-25T10:53:31.150020+0000","last_peered":"2024-04-25T10:53:31.150020+0000","last_clean":"2024-04-25T10:53:31.150020+0000","last_became_active":"2024-04-25T10:52:32.472174+0000","last_became_peered":"2024-04-25T10:52:32.472174+0000","last_unstale":"2024-04-25T10:53:31.150020+0000","last_undegraded":"2024-04-25T10:53:31.150020+0000","last_fullsized":"2024-04-25T10:53:31.150020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:09.473575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150137+0000","last_change":"2024-04-25T10:52:32.473617+0000","last_active":"2024-04-25T10:53:31.150137+0000","last_peered":"2024-04-25T10:53:31.150137+0000","last_clean":"2024-04-25T10:53:31.150137+0000","last_became_active":"2024-04-25T10:52:32.473261+0000","last_became_peered":"2024-04-25T10:52:32.473261+0000","last_unstale":"2024-04-25T10:53:31.150137+0000","last_undegraded":"2024-04-25T10:53:31.150137+0000","last_fullsized":"2024-04-25T10:53:31.150137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:05.568672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242750+0000","last_change":"2024-04-25T10:52:32.466296+0000","last_active":"2024-04-25T10:53:31.242750+0000","last_peered":"2024-04-25T10:53:31.242750+0000","last_clean":"2024-04-25T10:53:31.242750+0000","last_became_active":"2024-04-25T10:52:32.463153+0000","last_became_peered":"2024-04-25T10:52:32.463153+0000","last_unstale":"2024-04-25T10:53:31.242750+0000","last_undegraded":"2024-04-25T10:53:31.242750+0000","last_fullsized":"2024-04-25T10:53:31.242750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150155+0000","last_change":"2024-04-25T10:52:31.701949+0000","last_active":"2024-04-25T10:53:31.150155+0000","last_peered":"2024-04-25T10:53:31.150155+0000","last_clean":"2024-04-25T10:53:31.150155+0000","last_became_active":"2024-04-25T10:52:31.701508+0000","last_became_peered":"2024-04-25T10:52:31.701508+0000","last_unstale":"2024-04-25T10:53:31.150155+0000","last_undegraded":"2024-04-25T10:53:31.150155+0000","last_fullsized":"2024-04-25T10:53:31.150155+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:41.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153836+0000","last_change":"2024-04-25T10:52:32.468871+0000","last_active":"2024-04-25T10:53:31.153836+0000","last_peered":"2024-04-25T10:53:31.153836+0000","last_clean":"2024-04-25T10:53:31.153836+0000","last_became_active":"2024-04-25T10:52:32.468460+0000","last_became_peered":"2024-04-25T10:52:32.468460+0000","last_unstale":"2024-04-25T10:53:31.153836+0000","last_undegraded":"2024-04-25T10:53:31.153836+0000","last_fullsized":"2024-04-25T10:53:31.153836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:29.055639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154101+0000","last_change":"2024-04-25T10:52:31.705068+0000","last_active":"2024-04-25T10:53:31.154101+0000","last_peered":"2024-04-25T10:53:31.154101+0000","last_clean":"2024-04-25T10:53:31.154101+0000","last_became_active":"2024-04-25T10:52:31.704439+0000","last_became_peered":"2024-04-25T10:52:31.704439+0000","last_unstale":"2024-04-25T10:53:31.154101+0000","last_undegraded":"2024-04-25T10:53:31.154101+0000","last_fullsized":"2024-04-25T10:53:31.154101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.347653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300337+0000","last_change":"2024-04-25T10:52:32.476025+0000","last_active":"2024-04-25T10:53:31.300337+0000","last_peered":"2024-04-25T10:53:31.300337+0000","last_clean":"2024-04-25T10:53:31.300337+0000","last_became_active":"2024-04-25T10:52:32.475611+0000","last_became_peered":"2024-04-25T10:52:32.475611+0000","last_unstale":"2024-04-25T10:53:31.300337+0000","last_undegraded":"2024-04-25T10:53:31.300337+0000","last_fullsized":"2024-04-25T10:53:31.300337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:45.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147487+0000","last_change":"2024-04-25T10:52:31.703012+0000","last_active":"2024-04-25T10:53:31.147487+0000","last_peered":"2024-04-25T10:53:31.147487+0000","last_clean":"2024-04-25T10:53:31.147487+0000","last_became_active":"2024-04-25T10:52:31.702689+0000","last_became_peered":"2024-04-25T10:52:31.702689+0000","last_unstale":"2024-04-25T10:53:31.147487+0000","last_undegraded":"2024-04-25T10:53:31.147487+0000","last_fullsized":"2024-04-25T10:53:31.147487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:13.038859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146541+0000","last_change":"2024-04-25T10:52:32.475858+0000","last_active":"2024-04-25T10:53:31.146541+0000","last_peered":"2024-04-25T10:53:31.146541+0000","last_clean":"2024-04-25T10:53:31.146541+0000","last_became_active":"2024-04-25T10:52:32.475486+0000","last_became_peered":"2024-04-25T10:52:32.475486+0000","last_unstale":"2024-04-25T10:53:31.146541+0000","last_undegraded":"2024-04-25T10:53:31.146541+0000","last_fullsized":"2024-04-25T10:53:31.146541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.439208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.155911+0000","last_change":"2024-04-25T10:53:31.155911+0000","last_active":"2024-04-25T10:53:31.155911+0000","last_peered":"2024-04-25T10:53:31.155911+0000","last_clean":"2024-04-25T10:53:31.155911+0000","last_became_active":"2024-04-25T10:53:31.151778+0000","last_became_peered":"2024-04-25T10:53:31.151778+0000","last_unstale":"2024-04-25T10:53:31.155911+0000","last_undegraded":"2024-04-25T10:53:31.155911+0000","last_fullsized":"2024-04-25T10:53:31.155911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:46.936583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152237+0000","last_change":"2024-04-25T10:53:31.152237+0000","last_active":"2024-04-25T10:53:31.152237+0000","last_peered":"2024-04-25T10:53:31.152237+0000","last_clean":"2024-04-25T10:53:31.152237+0000","last_became_active":"2024-04-25T10:53:31.151821+0000","last_became_peered":"2024-04-25T10:53:31.151821+0000","last_unstale":"2024-04-25T10:53:31.152237+0000","last_undegraded":"2024-04-25T10:53:31.152237+0000","last_fullsized":"2024-04-25T10:53:31.152237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:02.453018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148877+0000","last_change":"2024-04-25T10:52:31.699644+0000","last_active":"2024-04-25T10:53:31.148877+0000","last_peered":"2024-04-25T10:53:31.148877+0000","last_clean":"2024-04-25T10:53:31.148877+0000","last_became_active":"2024-04-25T10:52:31.699315+0000","last_became_peered":"2024-04-25T10:52:31.699315+0000","last_unstale":"2024-04-25T10:53:31.148877+0000","last_undegraded":"2024-04-25T10:53:31.148877+0000","last_fullsized":"2024-04-25T10:53:31.148877+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:52.915096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147070+0000","last_change":"2024-04-25T10:52:32.463372+0000","last_active":"2024-04-25T10:53:31.147070+0000","last_peered":"2024-04-25T10:53:31.147070+0000","last_clean":"2024-04-25T10:53:31.147070+0000","last_became_active":"2024-04-25T10:52:32.462926+0000","last_became_peered":"2024-04-25T10:52:32.462926+0000","last_unstale":"2024-04-25T10:53:31.147070+0000","last_undegraded":"2024-04-25T10:53:31.147070+0000","last_fullsized":"2024-04-25T10:53:31.147070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:17.724268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243415+0000","last_change":"2024-04-25T10:52:31.695754+0000","last_active":"2024-04-25T10:53:31.243415+0000","last_peered":"2024-04-25T10:53:31.243415+0000","last_clean":"2024-04-25T10:53:31.243415+0000","last_became_active":"2024-04-25T10:52:31.695381+0000","last_became_peered":"2024-04-25T10:52:31.695381+0000","last_unstale":"2024-04-25T10:53:31.243415+0000","last_undegraded":"2024-04-25T10:53:31.243415+0000","last_fullsized":"2024-04-25T10:53:31.243415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:58.851534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154575+0000","last_change":"2024-04-25T10:53:31.154575+0000","last_active":"2024-04-25T10:53:31.154575+0000","last_peered":"2024-04-25T10:53:31.154575+0000","last_clean":"2024-04-25T10:53:31.154575+0000","last_became_active":"2024-04-25T10:53:31.154145+0000","last_became_peered":"2024-04-25T10:53:31.154145+0000","last_unstale":"2024-04-25T10:53:31.154575+0000","last_undegraded":"2024-04-25T10:53:31.154575+0000","last_fullsized":"2024-04-25T10:53:31.154575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:37.838587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145615+0000","last_change":"2024-04-25T10:52:31.695067+0000","last_active":"2024-04-25T10:53:31.145615+0000","last_peered":"2024-04-25T10:53:31.145615+0000","last_clean":"2024-04-25T10:53:31.145615+0000","last_became_active":"2024-04-25T10:52:31.694648+0000","last_became_peered":"2024-04-25T10:52:31.694648+0000","last_unstale":"2024-04-25T10:53:31.145615+0000","last_undegraded":"2024-04-25T10:53:31.145615+0000","last_fullsized":"2024-04-25T10:53:31.145615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:09.654835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152895+0000","last_change":"2024-04-25T10:52:32.453501+0000","last_active":"2024-04-25T10:53:31.152895+0000","last_peered":"2024-04-25T10:53:31.152895+0000","last_clean":"2024-04-25T10:53:31.152895+0000","last_became_active":"2024-04-25T10:52:32.453113+0000","last_became_peered":"2024-04-25T10:52:32.453113+0000","last_unstale":"2024-04-25T10:53:31.152895+0000","last_undegraded":"2024-04-25T10:53:31.152895+0000","last_fullsized":"2024-04-25T10:53:31.152895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:07.340709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148863+0000","last_change":"2024-04-25T10:52:31.697946+0000","last_active":"2024-04-25T10:53:31.148863+0000","last_peered":"2024-04-25T10:53:31.148863+0000","last_clean":"2024-04-25T10:53:31.148863+0000","last_became_active":"2024-04-25T10:52:31.697612+0000","last_became_peered":"2024-04-25T10:52:31.697612+0000","last_unstale":"2024-04-25T10:53:31.148863+0000","last_undegraded":"2024-04-25T10:53:31.148863+0000","last_fullsized":"2024-04-25T10:53:31.148863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.280776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150205+0000","last_change":"2024-04-25T10:52:32.473537+0000","last_active":"2024-04-25T10:53:31.150205+0000","last_peered":"2024-04-25T10:53:31.150205+0000","last_clean":"2024-04-25T10:53:31.150205+0000","last_became_active":"2024-04-25T10:52:32.473164+0000","last_became_peered":"2024-04-25T10:52:32.473164+0000","last_unstale":"2024-04-25T10:53:31.150205+0000","last_undegraded":"2024-04-25T10:53:31.150205+0000","last_fullsized":"2024-04-25T10:53:31.150205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.008031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244011+0000","last_change":"2024-04-25T10:52:31.694207+0000","last_active":"2024-04-25T10:53:31.244011+0000","last_peered":"2024-04-25T10:53:31.244011+0000","last_clean":"2024-04-25T10:53:31.244011+0000","last_became_active":"2024-04-25T10:52:31.693777+0000","last_became_peered":"2024-04-25T10:52:31.693777+0000","last_unstale":"2024-04-25T10:53:31.244011+0000","last_undegraded":"2024-04-25T10:53:31.244011+0000","last_fullsized":"2024-04-25T10:53:31.244011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:43.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146864+0000","last_change":"2024-04-25T10:52:32.475545+0000","last_active":"2024-04-25T10:53:31.146864+0000","last_peered":"2024-04-25T10:53:31.146864+0000","last_clean":"2024-04-25T10:53:31.146864+0000","last_became_active":"2024-04-25T10:52:32.475195+0000","last_became_peered":"2024-04-25T10:52:32.475195+0000","last_unstale":"2024-04-25T10:53:31.146864+0000","last_undegraded":"2024-04-25T10:53:31.146864+0000","last_fullsized":"2024-04-25T10:53:31.146864+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:49.900374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150024+0000","last_change":"2024-04-25T10:52:31.698184+0000","last_active":"2024-04-25T10:53:31.150024+0000","last_peered":"2024-04-25T10:53:31.150024+0000","last_clean":"2024-04-25T10:53:31.150024+0000","last_became_active":"2024-04-25T10:52:31.697816+0000","last_became_peered":"2024-04-25T10:52:31.697816+0000","last_unstale":"2024-04-25T10:53:31.150024+0000","last_undegraded":"2024-04-25T10:53:31.150024+0000","last_fullsized":"2024-04-25T10:53:31.150024+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:44.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300445+0000","last_change":"2024-04-25T10:52:32.472437+0000","last_active":"2024-04-25T10:53:31.300445+0000","last_peered":"2024-04-25T10:53:31.300445+0000","last_clean":"2024-04-25T10:53:31.300445+0000","last_became_active":"2024-04-25T10:52:32.471944+0000","last_became_peered":"2024-04-25T10:52:32.471944+0000","last_unstale":"2024-04-25T10:53:31.300445+0000","last_undegraded":"2024-04-25T10:53:31.300445+0000","last_fullsized":"2024-04-25T10:53:31.300445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:11.047158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149519+0000","last_change":"2024-04-25T10:52:31.696943+0000","last_active":"2024-04-25T10:53:31.149519+0000","last_peered":"2024-04-25T10:53:31.149519+0000","last_clean":"2024-04-25T10:53:31.149519+0000","last_became_active":"2024-04-25T10:52:31.696561+0000","last_became_peered":"2024-04-25T10:52:31.696561+0000","last_unstale":"2024-04-25T10:53:31.149519+0000","last_undegraded":"2024-04-25T10:53:31.149519+0000","last_fullsized":"2024-04-25T10:53:31.149519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:14.628800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146568+0000","last_change":"2024-04-25T10:52:32.461966+0000","last_active":"2024-04-25T10:53:31.146568+0000","last_peered":"2024-04-25T10:53:31.146568+0000","last_clean":"2024-04-25T10:53:31.146568+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:53:31.146568+0000","last_undegraded":"2024-04-25T10:53:31.146568+0000","last_fullsized":"2024-04-25T10:53:31.146568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'26","reported_seq":192,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:44.875184+0000","last_change":"2024-04-25T10:52:31.690938+0000","last_active":"2024-04-25T10:53:44.875184+0000","last_peered":"2024-04-25T10:53:44.875184+0000","last_clean":"2024-04-25T10:53:44.875184+0000","last_became_active":"2024-04-25T10:52:31.690511+0000","last_became_peered":"2024-04-25T10:52:31.690511+0000","last_unstale":"2024-04-25T10:53:44.875184+0000","last_undegraded":"2024-04-25T10:53:44.875184+0000","last_fullsized":"2024-04-25T10:53:44.875184+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:10.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":147,"num_read_kb":132,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147193+0000","last_change":"2024-04-25T10:52:32.474871+0000","last_active":"2024-04-25T10:53:31.147193+0000","last_peered":"2024-04-25T10:53:31.147193+0000","last_clean":"2024-04-25T10:53:31.147193+0000","last_became_active":"2024-04-25T10:52:32.474463+0000","last_became_peered":"2024-04-25T10:52:32.474463+0000","last_unstale":"2024-04-25T10:53:31.147193+0000","last_undegraded":"2024-04-25T10:53:31.147193+0000","last_fullsized":"2024-04-25T10:53:31.147193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:36.719230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'2","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242599+0000","last_change":"2024-04-25T10:52:31.691006+0000","last_active":"2024-04-25T10:53:31.242599+0000","last_peered":"2024-04-25T10:53:31.242599+0000","last_clean":"2024-04-25T10:53:31.242599+0000","last_became_active":"2024-04-25T10:52:31.690611+0000","last_became_peered":"2024-04-25T10:52:31.690611+0000","last_unstale":"2024-04-25T10:53:31.242599+0000","last_undegraded":"2024-04-25T10:53:31.242599+0000","last_fullsized":"2024-04-25T10:53:31.242599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:37.781160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145596+0000","last_change":"2024-04-25T10:52:32.463331+0000","last_active":"2024-04-25T10:53:31.145596+0000","last_peered":"2024-04-25T10:53:31.145596+0000","last_clean":"2024-04-25T10:53:31.145596+0000","last_became_active":"2024-04-25T10:52:32.462579+0000","last_became_peered":"2024-04-25T10:52:32.462579+0000","last_unstale":"2024-04-25T10:53:31.145596+0000","last_undegraded":"2024-04-25T10:53:31.145596+0000","last_fullsized":"2024-04-25T10:53:31.145596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148723+0000","last_change":"2024-04-25T10:52:31.704675+0000","last_active":"2024-04-25T10:53:31.148723+0000","last_peered":"2024-04-25T10:53:31.148723+0000","last_clean":"2024-04-25T10:53:31.148723+0000","last_became_active":"2024-04-25T10:52:31.704236+0000","last_became_peered":"2024-04-25T10:52:31.704236+0000","last_unstale":"2024-04-25T10:53:31.148723+0000","last_undegraded":"2024-04-25T10:53:31.148723+0000","last_fullsized":"2024-04-25T10:53:31.148723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.637006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150234+0000","last_change":"2024-04-25T10:52:32.462960+0000","last_active":"2024-04-25T10:53:31.150234+0000","last_peered":"2024-04-25T10:53:31.150234+0000","last_clean":"2024-04-25T10:53:31.150234+0000","last_became_active":"2024-04-25T10:52:32.461993+0000","last_became_peered":"2024-04-25T10:52:32.461993+0000","last_unstale":"2024-04-25T10:53:31.150234+0000","last_undegraded":"2024-04-25T10:53:31.150234+0000","last_fullsized":"2024-04-25T10:53:31.150234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:01.803209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243532+0000","last_change":"2024-04-25T10:52:31.698206+0000","last_active":"2024-04-25T10:53:31.243532+0000","last_peered":"2024-04-25T10:53:31.243532+0000","last_clean":"2024-04-25T10:53:31.243532+0000","last_became_active":"2024-04-25T10:52:31.697814+0000","last_became_peered":"2024-04-25T10:52:31.697814+0000","last_unstale":"2024-04-25T10:53:31.243532+0000","last_undegraded":"2024-04-25T10:53:31.243532+0000","last_fullsized":"2024-04-25T10:53:31.243532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:02.849229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149757+0000","last_change":"2024-04-25T10:52:32.473551+0000","last_active":"2024-04-25T10:53:31.149757+0000","last_peered":"2024-04-25T10:53:31.149757+0000","last_clean":"2024-04-25T10:53:31.149757+0000","last_became_active":"2024-04-25T10:52:32.473227+0000","last_became_peered":"2024-04-25T10:52:32.473227+0000","last_unstale":"2024-04-25T10:53:31.149757+0000","last_undegraded":"2024-04-25T10:53:31.149757+0000","last_fullsized":"2024-04-25T10:53:31.149757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:37.742066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'8","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146872+0000","last_change":"2024-04-25T10:52:31.700670+0000","last_active":"2024-04-25T10:53:31.146872+0000","last_peered":"2024-04-25T10:53:31.146872+0000","last_clean":"2024-04-25T10:53:31.146872+0000","last_became_active":"2024-04-25T10:52:31.700234+0000","last_became_peered":"2024-04-25T10:52:31.700234+0000","last_unstale":"2024-04-25T10:53:31.146872+0000","last_undegraded":"2024-04-25T10:53:31.146872+0000","last_fullsized":"2024-04-25T10:53:31.146872+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:21.795784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153193+0000","last_change":"2024-04-25T10:53:31.153193+0000","last_active":"2024-04-25T10:53:31.153193+0000","last_peered":"2024-04-25T10:53:31.153193+0000","last_clean":"2024-04-25T10:53:31.153193+0000","last_became_active":"2024-04-25T10:53:31.152723+0000","last_became_peered":"2024-04-25T10:53:31.152723+0000","last_unstale":"2024-04-25T10:53:31.153193+0000","last_undegraded":"2024-04-25T10:53:31.153193+0000","last_fullsized":"2024-04-25T10:53:31.153193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:42.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147445+0000","last_change":"2024-04-25T10:52:31.698507+0000","last_active":"2024-04-25T10:53:31.147445+0000","last_peered":"2024-04-25T10:53:31.147445+0000","last_clean":"2024-04-25T10:53:31.147445+0000","last_became_active":"2024-04-25T10:52:31.698148+0000","last_became_peered":"2024-04-25T10:52:31.698148+0000","last_unstale":"2024-04-25T10:53:31.147445+0000","last_undegraded":"2024-04-25T10:53:31.147445+0000","last_fullsized":"2024-04-25T10:53:31.147445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:48.993234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149622+0000","last_change":"2024-04-25T10:52:32.465874+0000","last_active":"2024-04-25T10:53:31.149622+0000","last_peered":"2024-04-25T10:53:31.149622+0000","last_clean":"2024-04-25T10:53:31.149622+0000","last_became_active":"2024-04-25T10:52:32.465511+0000","last_became_peered":"2024-04-25T10:52:32.465511+0000","last_unstale":"2024-04-25T10:53:31.149622+0000","last_undegraded":"2024-04-25T10:53:31.149622+0000","last_fullsized":"2024-04-25T10:53:31.149622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:15.274909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149935+0000","last_change":"2024-04-25T10:52:31.691074+0000","last_active":"2024-04-25T10:53:31.149935+0000","last_peered":"2024-04-25T10:53:31.149935+0000","last_clean":"2024-04-25T10:53:31.149935+0000","last_became_active":"2024-04-25T10:52:31.690402+0000","last_became_peered":"2024-04-25T10:52:31.690402+0000","last_unstale":"2024-04-25T10:53:31.149935+0000","last_undegraded":"2024-04-25T10:53:31.149935+0000","last_fullsized":"2024-04-25T10:53:31.149935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.529860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244120+0000","last_change":"2024-04-25T10:52:32.477968+0000","last_active":"2024-04-25T10:53:31.244120+0000","last_peered":"2024-04-25T10:53:31.244120+0000","last_clean":"2024-04-25T10:53:31.244120+0000","last_became_active":"2024-04-25T10:52:32.476395+0000","last_became_peered":"2024-04-25T10:52:32.476395+0000","last_unstale":"2024-04-25T10:53:31.244120+0000","last_undegraded":"2024-04-25T10:53:31.244120+0000","last_fullsized":"2024-04-25T10:53:31.244120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:13.128910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149922+0000","last_change":"2024-04-25T10:52:31.698934+0000","last_active":"2024-04-25T10:53:31.149922+0000","last_peered":"2024-04-25T10:53:31.149922+0000","last_clean":"2024-04-25T10:53:31.149922+0000","last_became_active":"2024-04-25T10:52:31.698565+0000","last_became_peered":"2024-04-25T10:52:31.698565+0000","last_unstale":"2024-04-25T10:53:31.149922+0000","last_undegraded":"2024-04-25T10:53:31.149922+0000","last_fullsized":"2024-04-25T10:53:31.149922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.360617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244087+0000","last_change":"2024-04-25T10:52:32.477538+0000","last_active":"2024-04-25T10:53:31.244087+0000","last_peered":"2024-04-25T10:53:31.244087+0000","last_clean":"2024-04-25T10:53:31.244087+0000","last_became_active":"2024-04-25T10:52:32.476088+0000","last_became_peered":"2024-04-25T10:52:32.476088+0000","last_unstale":"2024-04-25T10:53:31.244087+0000","last_undegraded":"2024-04-25T10:53:31.244087+0000","last_fullsized":"2024-04-25T10:53:31.244087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.987071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.244447+0000","last_change":"2024-04-25T10:53:31.244447+0000","last_active":"2024-04-25T10:53:31.244447+0000","last_peered":"2024-04-25T10:53:31.244447+0000","last_clean":"2024-04-25T10:53:31.244447+0000","last_became_active":"2024-04-25T10:53:31.243944+0000","last_became_peered":"2024-04-25T10:53:31.243944+0000","last_unstale":"2024-04-25T10:53:31.244447+0000","last_undegraded":"2024-04-25T10:53:31.244447+0000","last_fullsized":"2024-04-25T10:53:31.244447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:46.344278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145467+0000","last_change":"2024-04-25T10:52:31.706700+0000","last_active":"2024-04-25T10:53:31.145467+0000","last_peered":"2024-04-25T10:53:31.145467+0000","last_clean":"2024-04-25T10:53:31.145467+0000","last_became_active":"2024-04-25T10:52:31.706392+0000","last_became_peered":"2024-04-25T10:52:31.706392+0000","last_unstale":"2024-04-25T10:53:31.145467+0000","last_undegraded":"2024-04-25T10:53:31.145467+0000","last_fullsized":"2024-04-25T10:53:31.145467+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:58.867351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153145+0000","last_change":"2024-04-25T10:52:32.462603+0000","last_active":"2024-04-25T10:53:31.153145+0000","last_peered":"2024-04-25T10:53:31.153145+0000","last_clean":"2024-04-25T10:53:31.153145+0000","last_became_active":"2024-04-25T10:52:32.462238+0000","last_became_peered":"2024-04-25T10:52:32.462238+0000","last_unstale":"2024-04-25T10:53:31.153145+0000","last_undegraded":"2024-04-25T10:53:31.153145+0000","last_fullsized":"2024-04-25T10:53:31.153145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.555683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243644+0000","last_change":"2024-04-25T10:52:31.694054+0000","last_active":"2024-04-25T10:53:31.243644+0000","last_peered":"2024-04-25T10:53:31.243644+0000","last_clean":"2024-04-25T10:53:31.243644+0000","last_became_active":"2024-04-25T10:52:31.693693+0000","last_became_peered":"2024-04-25T10:52:31.693693+0000","last_unstale":"2024-04-25T10:53:31.243644+0000","last_undegraded":"2024-04-25T10:53:31.243644+0000","last_fullsized":"2024-04-25T10:53:31.243644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:22.238131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149828+0000","last_change":"2024-04-25T10:51:22.730925+0000","last_active":"2024-04-25T10:53:31.149828+0000","last_peered":"2024-04-25T10:53:31.149828+0000","last_clean":"2024-04-25T10:53:31.149828+0000","last_became_active":"2024-04-25T10:51:22.725168+0000","last_became_peered":"2024-04-25T10:51:22.725168+0000","last_unstale":"2024-04-25T10:53:31.149828+0000","last_undegraded":"2024-04-25T10:53:31.149828+0000","last_fullsized":"2024-04-25T10:53:31.149828+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_clean_scrub_stamp":"2024-04-25T10:47:56.758246+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:02.416333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145464+0000","last_change":"2024-04-25T10:52:32.461608+0000","last_active":"2024-04-25T10:53:31.145464+0000","last_peered":"2024-04-25T10:53:31.145464+0000","last_clean":"2024-04-25T10:53:31.145464+0000","last_became_active":"2024-04-25T10:52:32.461210+0000","last_became_peered":"2024-04-25T10:52:32.461210+0000","last_unstale":"2024-04-25T10:53:31.145464+0000","last_undegraded":"2024-04-25T10:53:31.145464+0000","last_fullsized":"2024-04-25T10:53:31.145464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:14.897053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150087+0000","last_change":"2024-04-25T10:52:31.701003+0000","last_active":"2024-04-25T10:53:31.150087+0000","last_peered":"2024-04-25T10:53:31.150087+0000","last_clean":"2024-04-25T10:53:31.150087+0000","last_became_active":"2024-04-25T10:52:31.699623+0000","last_became_peered":"2024-04-25T10:52:31.699623+0000","last_unstale":"2024-04-25T10:53:31.150087+0000","last_undegraded":"2024-04-25T10:53:31.150087+0000","last_fullsized":"2024-04-25T10:53:31.150087+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:45.175760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.147395+0000","last_change":"2024-04-25T10:52:31.702595+0000","last_active":"2024-04-25T10:53:31.147395+0000","last_peered":"2024-04-25T10:53:31.147395+0000","last_clean":"2024-04-25T10:53:31.147395+0000","last_became_active":"2024-04-25T10:52:31.702124+0000","last_became_peered":"2024-04-25T10:52:31.702124+0000","last_unstale":"2024-04-25T10:53:31.147395+0000","last_undegraded":"2024-04-25T10:53:31.147395+0000","last_fullsized":"2024-04-25T10:53:31.147395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:54.351859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145848+0000","last_change":"2024-04-25T10:52:32.467987+0000","last_active":"2024-04-25T10:53:31.145848+0000","last_peered":"2024-04-25T10:53:31.145848+0000","last_clean":"2024-04-25T10:53:31.145848+0000","last_became_active":"2024-04-25T10:52:32.467613+0000","last_became_peered":"2024-04-25T10:52:32.467613+0000","last_unstale":"2024-04-25T10:53:31.145848+0000","last_undegraded":"2024-04-25T10:53:31.145848+0000","last_fullsized":"2024-04-25T10:53:31.145848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:00.112102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'6","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150152+0000","last_change":"2024-04-25T10:52:31.699868+0000","last_active":"2024-04-25T10:53:31.150152+0000","last_peered":"2024-04-25T10:53:31.150152+0000","last_clean":"2024-04-25T10:53:31.150152+0000","last_became_active":"2024-04-25T10:52:31.699470+0000","last_became_peered":"2024-04-25T10:52:31.699470+0000","last_unstale":"2024-04-25T10:53:31.150152+0000","last_undegraded":"2024-04-25T10:53:31.150152+0000","last_fullsized":"2024-04-25T10:53:31.150152+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.161821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154082+0000","last_change":"2024-04-25T10:52:32.459082+0000","last_active":"2024-04-25T10:53:31.154082+0000","last_peered":"2024-04-25T10:53:31.154082+0000","last_clean":"2024-04-25T10:53:31.154082+0000","last_became_active":"2024-04-25T10:52:32.458190+0000","last_became_peered":"2024-04-25T10:52:32.458190+0000","last_unstale":"2024-04-25T10:53:31.154082+0000","last_undegraded":"2024-04-25T10:53:31.154082+0000","last_fullsized":"2024-04-25T10:53:31.154082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.280762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242887+0000","last_change":"2024-04-25T10:52:31.700636+0000","last_active":"2024-04-25T10:53:31.242887+0000","last_peered":"2024-04-25T10:53:31.242887+0000","last_clean":"2024-04-25T10:53:31.242887+0000","last_became_active":"2024-04-25T10:52:31.700268+0000","last_became_peered":"2024-04-25T10:52:31.700268+0000","last_unstale":"2024-04-25T10:53:31.242887+0000","last_undegraded":"2024-04-25T10:53:31.242887+0000","last_fullsized":"2024-04-25T10:53:31.242887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.439491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300393+0000","last_change":"2024-04-25T10:52:32.467406+0000","last_active":"2024-04-25T10:53:31.300393+0000","last_peered":"2024-04-25T10:53:31.300393+0000","last_clean":"2024-04-25T10:53:31.300393+0000","last_became_active":"2024-04-25T10:52:32.467053+0000","last_became_peered":"2024-04-25T10:52:32.467053+0000","last_unstale":"2024-04-25T10:53:31.300393+0000","last_undegraded":"2024-04-25T10:53:31.300393+0000","last_fullsized":"2024-04-25T10:53:31.300393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.301411+0000","last_change":"2024-04-25T10:53:31.301411+0000","last_active":"2024-04-25T10:53:31.301411+0000","last_peered":"2024-04-25T10:53:31.301411+0000","last_clean":"2024-04-25T10:53:31.301411+0000","last_became_active":"2024-04-25T10:53:31.301022+0000","last_became_peered":"2024-04-25T10:53:31.301022+0000","last_unstale":"2024-04-25T10:53:31.301411+0000","last_undegraded":"2024-04-25T10:53:31.301411+0000","last_fullsized":"2024-04-25T10:53:31.301411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:28.171444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146876+0000","last_change":"2024-04-25T10:52:32.472709+0000","last_active":"2024-04-25T10:53:31.146876+0000","last_peered":"2024-04-25T10:53:31.146876+0000","last_clean":"2024-04-25T10:53:31.146876+0000","last_became_active":"2024-04-25T10:52:32.472331+0000","last_became_peered":"2024-04-25T10:52:32.472331+0000","last_unstale":"2024-04-25T10:53:31.146876+0000","last_undegraded":"2024-04-25T10:53:31.146876+0000","last_fullsized":"2024-04-25T10:53:31.146876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.047682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153117+0000","last_change":"2024-04-25T10:52:31.697360+0000","last_active":"2024-04-25T10:53:31.153117+0000","last_peered":"2024-04-25T10:53:31.153117+0000","last_clean":"2024-04-25T10:53:31.153117+0000","last_became_active":"2024-04-25T10:52:31.696991+0000","last_became_peered":"2024-04-25T10:52:31.696991+0000","last_unstale":"2024-04-25T10:53:31.153117+0000","last_undegraded":"2024-04-25T10:53:31.153117+0000","last_fullsized":"2024-04-25T10:53:31.153117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:38.395372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150674+0000","last_change":"2024-04-25T10:52:32.472505+0000","last_active":"2024-04-25T10:53:31.150674+0000","last_peered":"2024-04-25T10:53:31.150674+0000","last_clean":"2024-04-25T10:53:31.150674+0000","last_became_active":"2024-04-25T10:52:32.472113+0000","last_became_peered":"2024-04-25T10:52:32.472113+0000","last_unstale":"2024-04-25T10:53:31.150674+0000","last_undegraded":"2024-04-25T10:53:31.150674+0000","last_fullsized":"2024-04-25T10:53:31.150674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:20.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242458+0000","last_change":"2024-04-25T10:52:31.696488+0000","last_active":"2024-04-25T10:53:31.242458+0000","last_peered":"2024-04-25T10:53:31.242458+0000","last_clean":"2024-04-25T10:53:31.242458+0000","last_became_active":"2024-04-25T10:52:31.696062+0000","last_became_peered":"2024-04-25T10:52:31.696062+0000","last_unstale":"2024-04-25T10:53:31.242458+0000","last_undegraded":"2024-04-25T10:53:31.242458+0000","last_fullsized":"2024-04-25T10:53:31.242458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:24.329938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.153458+0000","last_change":"2024-04-25T10:52:32.474555+0000","last_active":"2024-04-25T10:53:31.153458+0000","last_peered":"2024-04-25T10:53:31.153458+0000","last_clean":"2024-04-25T10:53:31.153458+0000","last_became_active":"2024-04-25T10:52:32.474001+0000","last_became_peered":"2024-04-25T10:52:32.474001+0000","last_unstale":"2024-04-25T10:53:31.153458+0000","last_undegraded":"2024-04-25T10:53:31.153458+0000","last_fullsized":"2024-04-25T10:53:31.153458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.775966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'2","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146979+0000","last_change":"2024-04-25T10:52:31.702251+0000","last_active":"2024-04-25T10:53:31.146979+0000","last_peered":"2024-04-25T10:53:31.146979+0000","last_clean":"2024-04-25T10:53:31.146979+0000","last_became_active":"2024-04-25T10:52:31.701763+0000","last_became_peered":"2024-04-25T10:52:31.701763+0000","last_unstale":"2024-04-25T10:53:31.146979+0000","last_undegraded":"2024-04-25T10:53:31.146979+0000","last_fullsized":"2024-04-25T10:53:31.146979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150054+0000","last_change":"2024-04-25T10:52:32.465494+0000","last_active":"2024-04-25T10:53:31.150054+0000","last_peered":"2024-04-25T10:53:31.150054+0000","last_clean":"2024-04-25T10:53:31.150054+0000","last_became_active":"2024-04-25T10:52:32.463855+0000","last_became_peered":"2024-04-25T10:52:32.463855+0000","last_unstale":"2024-04-25T10:53:31.150054+0000","last_undegraded":"2024-04-25T10:53:31.150054+0000","last_fullsized":"2024-04-25T10:53:31.150054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:40.970580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242816+0000","last_change":"2024-04-25T10:52:31.700749+0000","last_active":"2024-04-25T10:53:31.242816+0000","last_peered":"2024-04-25T10:53:31.242816+0000","last_clean":"2024-04-25T10:53:31.242816+0000","last_became_active":"2024-04-25T10:52:31.699109+0000","last_became_peered":"2024-04-25T10:52:31.699109+0000","last_unstale":"2024-04-25T10:53:31.242816+0000","last_undegraded":"2024-04-25T10:53:31.242816+0000","last_fullsized":"2024-04-25T10:53:31.242816+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:48.405333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150398+0000","last_change":"2024-04-25T10:52:32.468292+0000","last_active":"2024-04-25T10:53:31.150398+0000","last_peered":"2024-04-25T10:53:31.150398+0000","last_clean":"2024-04-25T10:53:31.150398+0000","last_became_active":"2024-04-25T10:52:32.467934+0000","last_became_peered":"2024-04-25T10:52:32.467934+0000","last_unstale":"2024-04-25T10:53:31.150398+0000","last_undegraded":"2024-04-25T10:53:31.150398+0000","last_fullsized":"2024-04-25T10:53:31.150398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.366735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149325+0000","last_change":"2024-04-25T10:52:31.705025+0000","last_active":"2024-04-25T10:53:31.149325+0000","last_peered":"2024-04-25T10:53:31.149325+0000","last_clean":"2024-04-25T10:53:31.149325+0000","last_became_active":"2024-04-25T10:52:31.704532+0000","last_became_peered":"2024-04-25T10:52:31.704532+0000","last_unstale":"2024-04-25T10:53:31.149325+0000","last_undegraded":"2024-04-25T10:53:31.149325+0000","last_fullsized":"2024-04-25T10:53:31.149325+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.371778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.158758+0000","last_change":"2024-04-25T10:53:31.158758+0000","last_active":"2024-04-25T10:53:31.158758+0000","last_peered":"2024-04-25T10:53:31.158758+0000","last_clean":"2024-04-25T10:53:31.158758+0000","last_became_active":"2024-04-25T10:53:31.152575+0000","last_became_peered":"2024-04-25T10:53:31.152575+0000","last_unstale":"2024-04-25T10:53:31.158758+0000","last_undegraded":"2024-04-25T10:53:31.158758+0000","last_fullsized":"2024-04-25T10:53:31.158758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145781+0000","last_change":"2024-04-25T10:52:31.705770+0000","last_active":"2024-04-25T10:53:31.145781+0000","last_peered":"2024-04-25T10:53:31.145781+0000","last_clean":"2024-04-25T10:53:31.145781+0000","last_became_active":"2024-04-25T10:52:31.705391+0000","last_became_peered":"2024-04-25T10:52:31.705391+0000","last_unstale":"2024-04-25T10:53:31.145781+0000","last_undegraded":"2024-04-25T10:53:31.145781+0000","last_fullsized":"2024-04-25T10:53:31.145781+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:32.831435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150206+0000","last_change":"2024-04-25T10:52:32.465550+0000","last_active":"2024-04-25T10:53:31.150206+0000","last_peered":"2024-04-25T10:53:31.150206+0000","last_clean":"2024-04-25T10:53:31.150206+0000","last_became_active":"2024-04-25T10:52:32.465034+0000","last_became_peered":"2024-04-25T10:52:32.465034+0000","last_unstale":"2024-04-25T10:53:31.150206+0000","last_undegraded":"2024-04-25T10:53:31.150206+0000","last_fullsized":"2024-04-25T10:53:31.150206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.810469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.150713+0000","last_change":"2024-04-25T10:52:32.470812+0000","last_active":"2024-04-25T10:53:31.150713+0000","last_peered":"2024-04-25T10:53:31.150713+0000","last_clean":"2024-04-25T10:53:31.150713+0000","last_became_active":"2024-04-25T10:52:32.469620+0000","last_became_peered":"2024-04-25T10:52:32.469620+0000","last_unstale":"2024-04-25T10:53:31.150713+0000","last_undegraded":"2024-04-25T10:53:31.150713+0000","last_fullsized":"2024-04-25T10:53:31.150713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242788+0000","last_change":"2024-04-25T10:52:32.464762+0000","last_active":"2024-04-25T10:53:31.242788+0000","last_peered":"2024-04-25T10:53:31.242788+0000","last_clean":"2024-04-25T10:53:31.242788+0000","last_became_active":"2024-04-25T10:52:32.464316+0000","last_became_peered":"2024-04-25T10:52:32.464316+0000","last_unstale":"2024-04-25T10:53:31.242788+0000","last_undegraded":"2024-04-25T10:53:31.242788+0000","last_fullsized":"2024-04-25T10:53:31.242788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:55.526572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.242433+0000","last_change":"2024-04-25T10:52:32.466429+0000","last_active":"2024-04-25T10:53:31.242433+0000","last_peered":"2024-04-25T10:53:31.242433+0000","last_clean":"2024-04-25T10:53:31.242433+0000","last_became_active":"2024-04-25T10:52:32.465132+0000","last_became_peered":"2024-04-25T10:52:32.465132+0000","last_unstale":"2024-04-25T10:53:31.242433+0000","last_undegraded":"2024-04-25T10:53:31.242433+0000","last_fullsized":"2024-04-25T10:53:31.242433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:14.274441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.300373+0000","last_change":"2024-04-25T10:52:32.464666+0000","last_active":"2024-04-25T10:53:31.300373+0000","last_peered":"2024-04-25T10:53:31.300373+0000","last_clean":"2024-04-25T10:53:31.300373+0000","last_became_active":"2024-04-25T10:52:32.464226+0000","last_became_peered":"2024-04-25T10:52:32.464226+0000","last_unstale":"2024-04-25T10:53:31.300373+0000","last_undegraded":"2024-04-25T10:53:31.300373+0000","last_fullsized":"2024-04-25T10:53:31.300373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:39.388542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145725+0000","last_change":"2024-04-25T10:52:32.465199+0000","last_active":"2024-04-25T10:53:31.145725+0000","last_peered":"2024-04-25T10:53:31.145725+0000","last_clean":"2024-04-25T10:53:31.145725+0000","last_became_active":"2024-04-25T10:52:32.464285+0000","last_became_peered":"2024-04-25T10:52:32.464285+0000","last_unstale":"2024-04-25T10:53:31.145725+0000","last_undegraded":"2024-04-25T10:53:31.145725+0000","last_fullsized":"2024-04-25T10:53:31.145725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.010994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149373+0000","last_change":"2024-04-25T10:52:32.469140+0000","last_active":"2024-04-25T10:53:31.149373+0000","last_peered":"2024-04-25T10:53:31.149373+0000","last_clean":"2024-04-25T10:53:31.149373+0000","last_became_active":"2024-04-25T10:52:32.468344+0000","last_became_peered":"2024-04-25T10:52:32.468344+0000","last_unstale":"2024-04-25T10:53:31.149373+0000","last_undegraded":"2024-04-25T10:53:31.149373+0000","last_fullsized":"2024-04-25T10:53:31.149373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:07.228059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.243075+0000","last_change":"2024-04-25T10:52:32.469326+0000","last_active":"2024-04-25T10:53:31.243075+0000","last_peered":"2024-04-25T10:53:31.243075+0000","last_clean":"2024-04-25T10:53:31.243075+0000","last_became_active":"2024-04-25T10:52:32.468905+0000","last_became_peered":"2024-04-25T10:52:32.468905+0000","last_unstale":"2024-04-25T10:53:31.243075+0000","last_undegraded":"2024-04-25T10:53:31.243075+0000","last_fullsized":"2024-04-25T10:53:31.243075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:31.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.154039+0000","last_change":"2024-04-25T10:52:32.459391+0000","last_active":"2024-04-25T10:53:31.154039+0000","last_peered":"2024-04-25T10:53:31.154039+0000","last_clean":"2024-04-25T10:53:31.154039+0000","last_became_active":"2024-04-25T10:52:32.458702+0000","last_became_peered":"2024-04-25T10:52:32.458702+0000","last_unstale":"2024-04-25T10:53:31.154039+0000","last_undegraded":"2024-04-25T10:53:31.154039+0000","last_fullsized":"2024-04-25T10:53:31.154039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.475714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.148497+0000","last_change":"2024-04-25T10:52:32.467395+0000","last_active":"2024-04-25T10:53:31.148497+0000","last_peered":"2024-04-25T10:53:31.148497+0000","last_clean":"2024-04-25T10:53:31.148497+0000","last_became_active":"2024-04-25T10:52:32.466976+0000","last_became_peered":"2024-04-25T10:52:32.466976+0000","last_unstale":"2024-04-25T10:53:31.148497+0000","last_undegraded":"2024-04-25T10:53:31.148497+0000","last_fullsized":"2024-04-25T10:53:31.148497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:22.011064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.152631+0000","last_change":"2024-04-25T10:52:32.458874+0000","last_active":"2024-04-25T10:53:31.152631+0000","last_peered":"2024-04-25T10:53:31.152631+0000","last_clean":"2024-04-25T10:53:31.152631+0000","last_became_active":"2024-04-25T10:52:32.458517+0000","last_became_peered":"2024-04-25T10:52:32.458517+0000","last_unstale":"2024-04-25T10:53:31.152631+0000","last_undegraded":"2024-04-25T10:53:31.152631+0000","last_fullsized":"2024-04-25T10:53:31.152631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:12.634521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.146425+0000","last_change":"2024-04-25T10:52:32.473068+0000","last_active":"2024-04-25T10:53:31.146425+0000","last_peered":"2024-04-25T10:53:31.146425+0000","last_clean":"2024-04-25T10:53:31.146425+0000","last_became_active":"2024-04-25T10:52:32.472713+0000","last_became_peered":"2024-04-25T10:52:32.472713+0000","last_unstale":"2024-04-25T10:53:31.146425+0000","last_undegraded":"2024-04-25T10:53:31.146425+0000","last_fullsized":"2024-04-25T10:53:31.146425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:43.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.145779+0000","last_change":"2024-04-25T10:52:32.468492+0000","last_active":"2024-04-25T10:53:31.145779+0000","last_peered":"2024-04-25T10:53:31.145779+0000","last_clean":"2024-04-25T10:53:31.145779+0000","last_became_active":"2024-04-25T10:52:32.468155+0000","last_became_peered":"2024-04-25T10:52:32.468155+0000","last_unstale":"2024-04-25T10:53:31.145779+0000","last_undegraded":"2024-04-25T10:53:31.145779+0000","last_fullsized":"2024-04-25T10:53:31.145779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.934230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:53:31.149774+0000","last_change":"2024-04-25T10:52:32.463057+0000","last_active":"2024-04-25T10:53:31.149774+0000","last_peered":"2024-04-25T10:53:31.149774+0000","last_clean":"2024-04-25T10:53:31.149774+0000","last_became_active":"2024-04-25T10:52:32.460983+0000","last_became_peered":"2024-04-25T10:52:32.460983+0000","last_unstale":"2024-04-25T10:53:31.149774+0000","last_undegraded":"2024-04-25T10:53:31.149774+0000","last_fullsized":"2024-04-25T10:53:31.149774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":153,"num_read_kb":138,"num_write":75,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":843776,"data_stored":411149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1523712,"data_stored":1062874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":411848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":411753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1445888,"data_stored":1002216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":847872,"data_stored":411266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":472195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":411239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":472229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":411370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":411171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1454080,"data_stored":1001816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:53:50.818 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T10:53:50.818 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:53:50.818 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:53:50.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T10:53:51.070 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:51.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:51 smithi019 ceph-mon[29209]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:53:51.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:51 smithi019 ceph-mon[29209]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:51.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:51 smithi112 ceph-mon[32799]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:53:51.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:51 smithi112 ceph-mon[32799]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:51.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:51 smithi169 ceph-mon[33049]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:53:51.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:51 smithi169 ceph-mon[33049]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.766+0000 7f679fca2700 1 -- 172.21.15.19:0/1171728070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6798104470 msgr2=0x7f6798104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.766+0000 7f679fca2700 1 --2- 172.21.15.19:0/1171728070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6798104470 0x7f6798104880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f67940099c0 tx=0x7f6794009cd0 comp rx=0 tx=0).stop 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.766+0000 7f679fca2700 1 -- 172.21.15.19:0/1171728070 shutdown_connections 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.766+0000 7f679fca2700 1 --2- 172.21.15.19:0/1171728070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67981096f0 0x7f6798111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.766+0000 7f679fca2700 1 --2- 172.21.15.19:0/1171728070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6798104470 0x7f6798104880 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.766+0000 7f679fca2700 1 --2- 172.21.15.19:0/1171728070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6798108da0 0x7f67981091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.766+0000 7f679fca2700 1 -- 172.21.15.19:0/1171728070 >> 172.21.15.19:0/1171728070 conn(0x7f67980ffd00 msgr2=0x7f6798102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.767+0000 7f679fca2700 1 -- 172.21.15.19:0/1171728070 shutdown_connections 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.767+0000 7f679fca2700 1 -- 172.21.15.19:0/1171728070 wait complete. 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.767+0000 7f679fca2700 1 Processor -- start 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.767+0000 7f679fca2700 1 -- start start 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679fca2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6798104470 0x7f67981a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679fca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6798108da0 0x7f67981a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679fca2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67981096f0 0x7f67981ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679fca2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6798114320 con 0x7f67981096f0 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679fca2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67981141a0 con 0x7f6798108da0 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679fca2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f67981144a0 con 0x7f6798104470 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67981096f0 0x7f67981ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67981096f0 0x7f67981ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51966/0 (socket says 172.21.15.19:51966) 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 -- 172.21.15.19:0/3367420137 learned_addr learned my addr 172.21.15.19:0/3367420137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679d23d700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6798108da0 0x7f67981a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679da3e700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6798104470 0x7f67981a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 -- 172.21.15.19:0/3367420137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6798104470 msgr2=0x7f67981a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6798104470 0x7f67981a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 -- 172.21.15.19:0/3367420137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6798108da0 msgr2=0x7f67981a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6798108da0 0x7f67981a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.768+0000 7f679e23f700 1 -- 172.21.15.19:0/3367420137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67981ad5c0 con 0x7f67981096f0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.769+0000 7f679e23f700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67981096f0 0x7f67981ad080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f679000ada0 tx=0x7f679000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.769+0000 7f678effd700 1 -- 172.21.15.19:0/3367420137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790017020 con 0x7f67981096f0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.769+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67981ad850 con 0x7f67981096f0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.769+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67981add60 con 0x7f67981096f0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.769+0000 7f678effd700 1 -- 172.21.15.19:0/3367420137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679000ce30 con 0x7f67981096f0 2024-04-25T10:53:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.769+0000 7f678effd700 1 -- 172.21.15.19:0/3367420137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790005350 con 0x7f67981096f0 2024-04-25T10:53:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.770+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c005330 con 0x7f67981096f0 2024-04-25T10:53:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.772+0000 7f678effd700 1 -- 172.21.15.19:0/3367420137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f679000c980 con 0x7f67981096f0 2024-04-25T10:53:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.773+0000 7f678effd700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67840afc90 0x7f67840b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.773+0000 7f679da3e700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67840afc90 0x7f67840b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.773+0000 7f678effd700 1 -- 172.21.15.19:0/3367420137 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f67900de3a0 con 0x7f67981096f0 2024-04-25T10:53:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.774+0000 7f679da3e700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67840afc90 0x7f67840b21a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6794003800 tx=0x7f679401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:52.778+0000 7f678effd700 1 -- 172.21.15.19:0/3367420137 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6790021020 con 0x7f67981096f0 2024-04-25T10:53:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.034+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f677c0058f0 con 0x7f67981096f0 2024-04-25T10:53:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.034+0000 7f678effd700 1 -- 172.21.15.19:0/3367420137 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f67900a6e60 con 0x7f67981096f0 2024-04-25T10:53:53.035 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:53:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67840afc90 msgr2=0x7f67840b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67840afc90 0x7f67840b21a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6794003800 tx=0x7f679401a040 comp rx=0 tx=0).stop 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67981096f0 msgr2=0x7f67981ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67981096f0 0x7f67981ad080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f679000ada0 tx=0x7f679000c330 comp rx=0 tx=0).stop 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 shutdown_connections 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f67840afc90 0x7f67840b21a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67981096f0 0x7f67981ad080 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6798104470 0x7f67981a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 --2- 172.21.15.19:0/3367420137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6798108da0 0x7f67981a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 >> 172.21.15.19:0/3367420137 conn(0x7f67980ffd00 msgr2=0x7f679810a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 shutdown_connections 2024-04-25T10:53:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.037+0000 7f679fca2700 1 -- 172.21.15.19:0/3367420137 wait complete. 2024-04-25T10:53:53.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:53 smithi019 ceph-mon[29209]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:53.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:53 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3367420137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T10:53:53.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:53 smithi112 ceph-mon[32799]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:53.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:53 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3367420137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T10:53:53.617 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:53:53.618 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:53:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 -- 172.21.15.19:0/437345648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8109a10 msgr2=0x7f36f810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 --2- 172.21.15.19:0/437345648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8109a10 0x7f36f810be30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009ca0 tx=0x7f36f002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 -- 172.21.15.19:0/437345648 shutdown_connections 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 --2- 172.21.15.19:0/437345648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8109a10 0x7f36f810be30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 --2- 172.21.15.19:0/437345648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36f8104d10 0x7f36f8105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 --2- 172.21.15.19:0/437345648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f8103ad0 0x7f36f81047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 -- 172.21.15.19:0/437345648 >> 172.21.15.19:0/437345648 conn(0x7f36f80fecc0 msgr2=0x7f36f8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 -- 172.21.15.19:0/437345648 shutdown_connections 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.733+0000 7f36fe605640 1 -- 172.21.15.19:0/437345648 wait complete. 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.734+0000 7f36fe605640 1 Processor -- start 2024-04-25T10:53:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.734+0000 7f36fe605640 1 -- start start 2024-04-25T10:53:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8103ad0 0x7f36f819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f8104d10 0x7f36f819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36f8109a10 0x7f36f81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36f810f600 con 0x7f36f8103ad0 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36f810f480 con 0x7f36f8104d10 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36f810f780 con 0x7f36f8109a10 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8103ad0 0x7f36f819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8103ad0 0x7f36f819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51992/0 (socket says 172.21.15.19:51992) 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 -- 172.21.15.19:0/3771893850 learned_addr learned my addr 172.21.15.19:0/3771893850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f77fe640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f8104d10 0x7f36f819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fcba2640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36f8109a10 0x7f36f81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 -- 172.21.15.19:0/3771893850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36f8109a10 msgr2=0x7f36f81a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36f8109a10 0x7f36f81a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 -- 172.21.15.19:0/3771893850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f8104d10 msgr2=0x7f36f819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f8104d10 0x7f36f819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 -- 172.21.15.19:0/3771893850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f81a1cc0 con 0x7f36f8103ad0 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f77fe640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f8104d10 0x7f36f819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f7fff640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8103ad0 0x7f36f819a180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f36e800d740 tx=0x7f36e800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f57fa640 1 -- 172.21.15.19:0/3771893850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8017070 con 0x7f36f8103ad0 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f57fa640 1 -- 172.21.15.19:0/3771893850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e800de50 con 0x7f36f8103ad0 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f81a1fb0 con 0x7f36f8103ad0 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36f57fa640 1 -- 172.21.15.19:0/3771893850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8003c20 con 0x7f36f8103ad0 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.735+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f81a2410 con 0x7f36f8103ad0 2024-04-25T10:53:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.737+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f8069310 con 0x7f36f8103ad0 2024-04-25T10:53:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.739+0000 7f36f57fa640 1 -- 172.21.15.19:0/3771893850 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f36e8003dc0 con 0x7f36f8103ad0 2024-04-25T10:53:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.740+0000 7f36f57fa640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f36e40afb80 0x7f36e40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.740+0000 7f36f77fe640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f36e40afb80 0x7f36e40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.740+0000 7f36f57fa640 1 -- 172.21.15.19:0/3771893850 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f36e80ded10 con 0x7f36f8103ad0 2024-04-25T10:53:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.740+0000 7f36f77fe640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f36e40afb80 0x7f36e40b2090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f36f819b220 tx=0x7f36e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.743+0000 7f36f57fa640 1 -- 172.21.15.19:0/3771893850 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36e80a7740 con 0x7f36f8103ad0 2024-04-25T10:53:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:53 smithi169 ceph-mon[33049]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:53 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3367420137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T10:53:53.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.933+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36f802dc10 con 0x7f36f8103ad0 2024-04-25T10:53:53.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.935+0000 7f36f57fa640 1 -- 172.21.15.19:0/3771893850 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7f36e80a7040 con 0x7f36f8103ad0 2024-04-25T10:53:53.936 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:53.936 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:53:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f36e40afb80 msgr2=0x7f36e40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f36e40afb80 0x7f36e40b2090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f36f819b220 tx=0x7f36e0005f90 comp rx=0 tx=0).stop 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8103ad0 msgr2=0x7f36f819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8103ad0 0x7f36f819a180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f36e800d740 tx=0x7f36e800dc00 comp rx=0 tx=0).stop 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 shutdown_connections 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36f8109a10 0x7f36f81a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f8104d10 0x7f36f819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f36e40afb80 0x7f36e40b2090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 --2- 172.21.15.19:0/3771893850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f8103ad0 0x7f36f819a180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 >> 172.21.15.19:0/3771893850 conn(0x7f36f80fecc0 msgr2=0x7f36f81059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 shutdown_connections 2024-04-25T10:53:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:53.937+0000 7f36fe605640 1 -- 172.21.15.19:0/3771893850 wait complete. 2024-04-25T10:53:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:53:53.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:53:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 -- 172.21.15.19:0/2005268042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038108930 msgr2=0x7fe03810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/2005268042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038108930 0x7fe03810ad50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe030009ca0 tx=0x7fe03002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 -- 172.21.15.19:0/2005268042 shutdown_connections 2024-04-25T10:53:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/2005268042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038108930 0x7fe03810ad50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/2005268042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe038103c30 0x7fe038104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/2005268042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0381029f0 0x7fe0381036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 -- 172.21.15.19:0/2005268042 >> 172.21.15.19:0/2005268042 conn(0x7fe0380fdbc0 msgr2=0x7fe038100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 -- 172.21.15.19:0/2005268042 shutdown_connections 2024-04-25T10:53:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.065+0000 7fe03ceb6640 1 -- 172.21.15.19:0/2005268042 wait complete. 2024-04-25T10:53:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 Processor -- start 2024-04-25T10:53:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 -- start start 2024-04-25T10:53:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0381029f0 0x7fe03819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038103c30 0x7fe03819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe038108930 0x7fe0381a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810f620 con 0x7fe038103c30 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810f4a0 con 0x7fe038108930 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03ceb6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810f7a0 con 0x7fe0381029f0 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe035d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038103c30 0x7fe03819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe035d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038103c30 0x7fe03819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52010/0 (socket says 172.21.15.19:52010) 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe035d9b640 1 -- 172.21.15.19:0/921660237 learned_addr learned my addr 172.21.15.19:0/921660237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.066+0000 7fe03659c640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0381029f0 0x7fe03819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe036d9d640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe038108930 0x7fe0381a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe035d9b640 1 -- 172.21.15.19:0/921660237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0381029f0 msgr2=0x7fe03819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe035d9b640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0381029f0 0x7fe03819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe035d9b640 1 -- 172.21.15.19:0/921660237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe038108930 msgr2=0x7fe0381a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe035d9b640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe038108930 0x7fe0381a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe035d9b640 1 -- 172.21.15.19:0/921660237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0381a1d50 con 0x7fe038103c30 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe036d9d640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe038108930 0x7fe0381a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe035d9b640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038103c30 0x7fe03819a7c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00d950 tx=0x7fe02c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe0277fe640 1 -- 172.21.15.19:0/921660237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c017070 con 0x7fe038103c30 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe0277fe640 1 -- 172.21.15.19:0/921660237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02c009c30 con 0x7fe038103c30 2024-04-25T10:53:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0381a1fe0 con 0x7fe038103c30 2024-04-25T10:53:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe0277fe640 1 -- 172.21.15.19:0/921660237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00f620 con 0x7fe038103c30 2024-04-25T10:53:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.067+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0381a2520 con 0x7fe038103c30 2024-04-25T10:53:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.068+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe038107740 con 0x7fe038103c30 2024-04-25T10:53:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.069+0000 7fe0277fe640 1 -- 172.21.15.19:0/921660237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe02c010460 con 0x7fe038103c30 2024-04-25T10:53:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.070+0000 7fe0277fe640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe0200afc00 0x7fe0200b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.070+0000 7fe0277fe640 1 -- 172.21.15.19:0/921660237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fe02c0dea30 con 0x7fe038103c30 2024-04-25T10:53:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.070+0000 7fe03659c640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe0200afc00 0x7fe0200b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.071+0000 7fe03659c640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe0200afc00 0x7fe0200b2110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0280058c0 tx=0x7fe02800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:54.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.075+0000 7fe0277fe640 1 -- 172.21.15.19:0/921660237 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe02c0a74c0 con 0x7fe038103c30 2024-04-25T10:53:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.265+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0380637a0 con 0x7fe038103c30 2024-04-25T10:53:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.267+0000 7fe0277fe640 1 -- 172.21.15.19:0/921660237 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fe02c009de0 con 0x7fe038103c30 2024-04-25T10:53:54.268 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:53:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe0200afc00 msgr2=0x7fe0200b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe0200afc00 0x7fe0200b2110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0280058c0 tx=0x7fe02800a4c0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038103c30 msgr2=0x7fe03819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038103c30 0x7fe03819a7c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00d950 tx=0x7fe02c00de10 comp rx=0 tx=0).stop 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 shutdown_connections 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe038108930 0x7fe0381a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fe0200afc00 0x7fe0200b2110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe038103c30 0x7fe03819a7c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 --2- 172.21.15.19:0/921660237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0381029f0 0x7fe03819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.269+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 >> 172.21.15.19:0/921660237 conn(0x7fe0380fdbc0 msgr2=0x7fe038104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.270+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 shutdown_connections 2024-04-25T10:53:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.270+0000 7fe03ceb6640 1 -- 172.21.15.19:0/921660237 wait complete. 2024-04-25T10:53:54.282 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:53:54.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.395+0000 7fcdaf80c640 1 -- 172.21.15.19:0/2891881206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8109790 msgr2=0x7fcda810bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.395+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/2891881206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8109790 0x7fcda810bca0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009ca0 tx=0x7fcda002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 -- 172.21.15.19:0/2891881206 shutdown_connections 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/2891881206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8109790 0x7fcda810bca0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/2891881206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8106c20 0x7fcda8109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/2891881206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda8101e40 0x7fcda8102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 -- 172.21.15.19:0/2891881206 >> 172.21.15.19:0/2891881206 conn(0x7fcda80fd350 msgr2=0x7fcda80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 -- 172.21.15.19:0/2891881206 shutdown_connections 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 -- 172.21.15.19:0/2891881206 wait complete. 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 Processor -- start 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.396+0000 7fcdaf80c640 1 -- start start 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8101e40 0x7fcda819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8106c20 0x7fcda819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda8109790 0x7fcda81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810f600 con 0x7fcda8101e40 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810f480 con 0x7fcda8106c20 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810f780 con 0x7fcda8109790 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8101e40 0x7fcda819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8101e40 0x7fcda819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52028/0 (socket says 172.21.15.19:52028) 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 -- 172.21.15.19:0/258028762 learned_addr learned my addr 172.21.15.19:0/258028762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdacda7640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8106c20 0x7fcda819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdadda9640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda8109790 0x7fcda81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 -- 172.21.15.19:0/258028762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda8109790 msgr2=0x7fcda81a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda8109790 0x7fcda81a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 -- 172.21.15.19:0/258028762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8106c20 msgr2=0x7fcda819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8106c20 0x7fcda819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 -- 172.21.15.19:0/258028762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda81a1cc0 con 0x7fcda8101e40 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdadda9640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda8109790 0x7fcda81a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdad5a8640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8101e40 0x7fcda819a180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcda400d770 tx=0x7fcda400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcd9e7fc640 1 -- 172.21.15.19:0/258028762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4017070 con 0x7fcda8101e40 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda81a1fb0 con 0x7fcda8101e40 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcd9e7fc640 1 -- 172.21.15.19:0/258028762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda4009c30 con 0x7fcda8101e40 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcd9e7fc640 1 -- 172.21.15.19:0/258028762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda400f620 con 0x7fcda8101e40 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.397+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda81a2470 con 0x7fcda8101e40 2024-04-25T10:53:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.398+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda80fee70 con 0x7fcda8101e40 2024-04-25T10:53:54.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.400+0000 7fcd9e7fc640 1 -- 172.21.15.19:0/258028762 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcda401a070 con 0x7fcda8101e40 2024-04-25T10:53:54.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.400+0000 7fcd9e7fc640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcd940afb80 0x7fcd940b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.401+0000 7fcdacda7640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcd940afb80 0x7fcd940b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.401+0000 7fcd9e7fc640 1 -- 172.21.15.19:0/258028762 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fcda40de970 con 0x7fcda8101e40 2024-04-25T10:53:54.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.401+0000 7fcdacda7640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcd940afb80 0x7fcd940b2090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd98005fd0 tx=0x7fcd98005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:54.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.404+0000 7fcd9e7fc640 1 -- 172.21.15.19:0/258028762 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcda4009de0 con 0x7fcda8101e40 2024-04-25T10:53:54.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:54 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3771893850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:54.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.591+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcda802dce0 con 0x7fcda8101e40 2024-04-25T10:53:54.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.592+0000 7fcd9e7fc640 1 -- 172.21.15.19:0/258028762 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fcda40a7400 con 0x7fcda8101e40 2024-04-25T10:53:54.593 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcd940afb80 msgr2=0x7fcd940b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcd940afb80 0x7fcd940b2090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd98005fd0 tx=0x7fcd98005850 comp rx=0 tx=0).stop 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8101e40 msgr2=0x7fcda819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8101e40 0x7fcda819a180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcda400d770 tx=0x7fcda400dc30 comp rx=0 tx=0).stop 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 shutdown_connections 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcda8109790 0x7fcda81a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8106c20 0x7fcda819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcd940afb80 0x7fcd940b2090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 --2- 172.21.15.19:0/258028762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda8101e40 0x7fcda819a180 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 >> 172.21.15.19:0/258028762 conn(0x7fcda80fd350 msgr2=0x7fcda8107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 shutdown_connections 2024-04-25T10:53:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.594+0000 7fcdaf80c640 1 -- 172.21.15.19:0/258028762 wait complete. 2024-04-25T10:53:54.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:54 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3771893850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:54.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 -- 172.21.15.19:0/2130315835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1029e0 msgr2=0x7f4eac1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 --2- 172.21.15.19:0/2130315835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1029e0 0x7f4eac1036e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009ca0 tx=0x7f4ea402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 -- 172.21.15.19:0/2130315835 shutdown_connections 2024-04-25T10:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 --2- 172.21.15.19:0/2130315835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac108920 0x7f4eac10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 --2- 172.21.15.19:0/2130315835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eac103c20 0x7f4eac104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 --2- 172.21.15.19:0/2130315835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1029e0 0x7f4eac1036e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 -- 172.21.15.19:0/2130315835 >> 172.21.15.19:0/2130315835 conn(0x7f4eac0fdbd0 msgr2=0x7f4eac100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 -- 172.21.15.19:0/2130315835 shutdown_connections 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 -- 172.21.15.19:0/2130315835 wait complete. 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 Processor -- start 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.720+0000 7f4eb313f640 1 -- start start 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb313f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eac1029e0 0x7f4eac199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb313f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac103c20 0x7f4eac199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb313f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac108920 0x7f4eac1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb313f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10e4c0 con 0x7f4eac108920 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb313f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10e340 con 0x7f4eac103c20 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb313f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10e640 con 0x7f4eac1029e0 2024-04-25T10:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac108920 0x7f4eac1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac108920 0x7f4eac1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52040/0 (socket says 172.21.15.19:52040) 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 -- 172.21.15.19:0/1321862678 learned_addr learned my addr 172.21.15.19:0/1321862678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eabfff640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac103c20 0x7f4eac199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb0edb640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eac1029e0 0x7f4eac199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 -- 172.21.15.19:0/1321862678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eac1029e0 msgr2=0x7f4eac199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eac1029e0 0x7f4eac199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 -- 172.21.15.19:0/1321862678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac103c20 msgr2=0x7f4eac199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac103c20 0x7f4eac199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 -- 172.21.15.19:0/1321862678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac1a0eb0 con 0x7f4eac108920 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb16dc640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac108920 0x7f4eac1a08f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400d750 tx=0x7f4e9400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4ea9ffb640 1 -- 172.21.15.19:0/1321862678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94017070 con 0x7f4eac108920 2024-04-25T10:53:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4ea9ffb640 1 -- 172.21.15.19:0/1321862678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e940097c0 con 0x7f4eac108920 2024-04-25T10:53:54.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac1a11a0 con 0x7f4eac108920 2024-04-25T10:53:54.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.721+0000 7f4ea9ffb640 1 -- 172.21.15.19:0/1321862678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94004ba0 con 0x7f4eac108920 2024-04-25T10:53:54.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.722+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac1a16f0 con 0x7f4eac108920 2024-04-25T10:53:54.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.723+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eac107730 con 0x7f4eac108920 2024-04-25T10:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.724+0000 7f4ea9ffb640 1 -- 172.21.15.19:0/1321862678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4e94009c90 con 0x7f4eac108920 2024-04-25T10:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.725+0000 7f4ea9ffb640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4e980afc00 0x7f4e980b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.725+0000 7f4eb0edb640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4e980afc00 0x7f4e980b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.725+0000 7f4ea9ffb640 1 -- 172.21.15.19:0/1321862678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f4e940de7d0 con 0x7f4eac108920 2024-04-25T10:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.725+0000 7f4eb0edb640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4e980afc00 0x7f4e980b2110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40094e0 tx=0x7f4ea4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:54.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.730+0000 7f4ea9ffb640 1 -- 172.21.15.19:0/1321862678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e940a7200 con 0x7f4eac108920 2024-04-25T10:53:54.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:54 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3771893850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:54.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.919+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4eac02ddc0 con 0x7f4eac108920 2024-04-25T10:53:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.919+0000 7f4ea9ffb640 1 -- 172.21.15.19:0/1321862678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f4e940a6b00 con 0x7f4eac108920 2024-04-25T10:53:54.920 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:53:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4e980afc00 msgr2=0x7f4e980b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4e980afc00 0x7f4e980b2110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40094e0 tx=0x7f4ea4005fd0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac108920 msgr2=0x7f4eac1a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac108920 0x7f4eac1a08f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400d750 tx=0x7f4e9400db50 comp rx=0 tx=0).stop 2024-04-25T10:53:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 shutdown_connections 2024-04-25T10:53:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4e980afc00 0x7f4e980b2110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac108920 0x7f4eac1a08f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac103c20 0x7f4eac199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 --2- 172.21.15.19:0/1321862678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eac1029e0 0x7f4eac199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 >> 172.21.15.19:0/1321862678 conn(0x7f4eac0fdbd0 msgr2=0x7f4eac104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 shutdown_connections 2024-04-25T10:53:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:54.921+0000 7f4eb313f640 1 -- 172.21.15.19:0/1321862678 wait complete. 2024-04-25T10:53:54.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:53:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 -- 172.21.15.19:0/3557407587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88104250 msgr2=0x7f0b88106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 --2- 172.21.15.19:0/3557407587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88104250 0x7f0b88106670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009900 tx=0x7f0b7802a410 comp rx=0 tx=0).stop 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 -- 172.21.15.19:0/3557407587 shutdown_connections 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 --2- 172.21.15.19:0/3557407587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88109750 0x7f0b8810bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 --2- 172.21.15.19:0/3557407587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b88106c40 0x7f0b881090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 --2- 172.21.15.19:0/3557407587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88104250 0x7f0b88106670 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 -- 172.21.15.19:0/3557407587 >> 172.21.15.19:0/3557407587 conn(0x7f0b880fdbd0 msgr2=0x7f0b88100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 -- 172.21.15.19:0/3557407587 shutdown_connections 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.044+0000 7f0b8d849640 1 -- 172.21.15.19:0/3557407587 wait complete. 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 Processor -- start 2024-04-25T10:53:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 -- start start 2024-04-25T10:53:55.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88104250 0x7f0b8819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88106c40 0x7f0b8819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b88109750 0x7f0b881a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8810f5c0 con 0x7f0b88106c40 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8810f440 con 0x7f0b88104250 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b8d849640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8810f740 con 0x7f0b88109750 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88106c40 0x7f0b8819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88106c40 0x7f0b8819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52058/0 (socket says 172.21.15.19:52058) 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.045+0000 7f0b867fc640 1 -- 172.21.15.19:0/903143069 learned_addr learned my addr 172.21.15.19:0/903143069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b877fe640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b88109750 0x7f0b881a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b86ffd640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88104250 0x7f0b8819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b867fc640 1 -- 172.21.15.19:0/903143069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b88109750 msgr2=0x7f0b881a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b867fc640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b88109750 0x7f0b881a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b867fc640 1 -- 172.21.15.19:0/903143069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88104250 msgr2=0x7f0b8819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b867fc640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88104250 0x7f0b8819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b867fc640 1 -- 172.21.15.19:0/903143069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b881a1d40 con 0x7f0b88106c40 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b86ffd640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88104250 0x7f0b8819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b867fc640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88106c40 0x7f0b8819a740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d770 tx=0x7f0b7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b8c847640 1 -- 172.21.15.19:0/903143069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c017070 con 0x7f0b88106c40 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b8c847640 1 -- 172.21.15.19:0/903143069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c0054d0 con 0x7f0b88106c40 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881a1fd0 con 0x7f0b88106c40 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b8c847640 1 -- 172.21.15.19:0/903143069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004c30 con 0x7f0b88106c40 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.046+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881a2470 con 0x7f0b88106c40 2024-04-25T10:53:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.047+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b880ff6f0 con 0x7f0b88106c40 2024-04-25T10:53:55.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.049+0000 7f0b8c847640 1 -- 172.21.15.19:0/903143069 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0b7c004dd0 con 0x7f0b88106c40 2024-04-25T10:53:55.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.049+0000 7f0b8c847640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0b700afb80 0x7f0b700b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.049+0000 7f0b86ffd640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0b700afb80 0x7f0b700b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.049+0000 7f0b8c847640 1 -- 172.21.15.19:0/903143069 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f0b7c0decb0 con 0x7f0b88106c40 2024-04-25T10:53:55.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.050+0000 7f0b86ffd640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0b700afb80 0x7f0b700b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0b78000c00 tx=0x7f0b780314c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:55.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.053+0000 7f0b8c847640 1 -- 172.21.15.19:0/903143069 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b7c0a7740 con 0x7f0b88106c40 2024-04-25T10:53:55.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.278+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b8802db50 con 0x7f0b88106c40 2024-04-25T10:53:55.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.279+0000 7f0b8c847640 1 -- 172.21.15.19:0/903143069 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f0b7c0a7040 con 0x7f0b88106c40 2024-04-25T10:53:55.280 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:55.280 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:53:55.280 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:53:55.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0b700afb80 msgr2=0x7f0b700b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0b700afb80 0x7f0b700b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0b78000c00 tx=0x7f0b780314c0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88106c40 msgr2=0x7f0b8819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88106c40 0x7f0b8819a740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d770 tx=0x7f0b7c00dc30 comp rx=0 tx=0).stop 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 shutdown_connections 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b88109750 0x7f0b881a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0b700afb80 0x7f0b700b2090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b88106c40 0x7f0b8819a740 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 --2- 172.21.15.19:0/903143069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88104250 0x7f0b8819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.281+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 >> 172.21.15.19:0/903143069 conn(0x7f0b880fdbd0 msgr2=0x7f0b88104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.282+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 shutdown_connections 2024-04-25T10:53:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.282+0000 7f0b8d849640 1 -- 172.21.15.19:0/903143069 wait complete. 2024-04-25T10:53:55.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:53:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 -- 172.21.15.19:0/580036413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c103b00 msgr2=0x7f2b9c104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 --2- 172.21.15.19:0/580036413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c103b00 0x7f2b9c104800 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009900 tx=0x7f2b8c02a410 comp rx=0 tx=0).stop 2024-04-25T10:53:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 -- 172.21.15.19:0/580036413 shutdown_connections 2024-04-25T10:53:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 --2- 172.21.15.19:0/580036413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b9c109a40 0x7f2b9c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 --2- 172.21.15.19:0/580036413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b9c104d40 0x7f2b9c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 --2- 172.21.15.19:0/580036413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c103b00 0x7f2b9c104800 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 -- 172.21.15.19:0/580036413 >> 172.21.15.19:0/580036413 conn(0x7f2b9c0fed10 msgr2=0x7f2b9c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:55.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 -- 172.21.15.19:0/580036413 shutdown_connections 2024-04-25T10:53:55.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.405+0000 7f2ba106e640 1 -- 172.21.15.19:0/580036413 wait complete. 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 Processor -- start 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 -- start start 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b9c103b00 0x7f2b9c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c104d40 0x7f2b9c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b9c109a40 0x7f2b9c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c10f660 con 0x7f2b9c104d40 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c10f4e0 con 0x7f2b9c103b00 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2ba106e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c10f7e0 con 0x7f2b9c109a40 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2b9a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c104d40 0x7f2b9c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2b9a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c104d40 0x7f2b9c19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52076/0 (socket says 172.21.15.19:52076) 2024-04-25T10:53:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.406+0000 7f2b9a59c640 1 -- 172.21.15.19:0/2616721103 learned_addr learned my addr 172.21.15.19:0/2616721103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9ad9d640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b9c103b00 0x7f2b9c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9b59e640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b9c109a40 0x7f2b9c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9a59c640 1 -- 172.21.15.19:0/2616721103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b9c109a40 msgr2=0x7f2b9c1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9a59c640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b9c109a40 0x7f2b9c1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9a59c640 1 -- 172.21.15.19:0/2616721103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b9c103b00 msgr2=0x7f2b9c19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9a59c640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b9c103b00 0x7f2b9c19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9a59c640 1 -- 172.21.15.19:0/2616721103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c1a1e00 con 0x7f2b9c104d40 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9ad9d640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b9c103b00 0x7f2b9c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b9a59c640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c104d40 0x7f2b9c19a800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b4a0 tx=0x7f2b9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b83fff640 1 -- 172.21.15.19:0/2616721103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90014070 con 0x7f2b9c104d40 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b83fff640 1 -- 172.21.15.19:0/2616721103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b90010ac0 con 0x7f2b9c104d40 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9c1a20f0 con 0x7f2b9c104d40 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2b83fff640 1 -- 172.21.15.19:0/2616721103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90017c30 con 0x7f2b9c104d40 2024-04-25T10:53:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.407+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9c1a2710 con 0x7f2b9c104d40 2024-04-25T10:53:55.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.408+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b9c108850 con 0x7f2b9c104d40 2024-04-25T10:53:55.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.410+0000 7f2b83fff640 1 -- 172.21.15.19:0/2616721103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2b90017070 con 0x7f2b9c104d40 2024-04-25T10:53:55.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.411+0000 7f2b83fff640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2b840afc00 0x7f2b840b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.411+0000 7f2b9ad9d640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2b840afc00 0x7f2b840b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.411+0000 7f2b83fff640 1 -- 172.21.15.19:0/2616721103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f2b900de3e0 con 0x7f2b9c104d40 2024-04-25T10:53:55.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.411+0000 7f2b9ad9d640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2b840afc00 0x7f2b840b2110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009230 tx=0x7f2b8c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:55.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.415+0000 7f2b83fff640 1 -- 172.21.15.19:0/2616721103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b900a6f80 con 0x7f2b9c104d40 2024-04-25T10:53:55.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:55 smithi112 ceph-mon[32799]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:55.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:55 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/921660237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:55 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/258028762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:55 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1321862678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.602+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b9c02dd20 con 0x7f2b9c104d40 2024-04-25T10:53:55.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.604+0000 7f2b83fff640 1 -- 172.21.15.19:0/2616721103 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7f2b900a6880 con 0x7f2b9c104d40 2024-04-25T10:53:55.605 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:55.605 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2b840afc00 msgr2=0x7f2b840b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2b840afc00 0x7f2b840b2110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009230 tx=0x7f2b8c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c104d40 msgr2=0x7f2b9c19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c104d40 0x7f2b9c19a800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b4a0 tx=0x7f2b9000b960 comp rx=0 tx=0).stop 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 shutdown_connections 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b9c109a40 0x7f2b9c1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2b840afc00 0x7f2b840b2110 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9c104d40 0x7f2b9c19a800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 --2- 172.21.15.19:0/2616721103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b9c103b00 0x7f2b9c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 >> 172.21.15.19:0/2616721103 conn(0x7f2b9c0fed10 msgr2=0x7f2b9c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 shutdown_connections 2024-04-25T10:53:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.606+0000 7f2ba106e640 1 -- 172.21.15.19:0/2616721103 wait complete. 2024-04-25T10:53:55.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:55 smithi019 ceph-mon[29209]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:55 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/921660237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:55 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/258028762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:55 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1321862678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:53:55.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 -- 172.21.15.19:0/2159901135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108920 msgr2=0x7f7fec10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 --2- 172.21.15.19:0/2159901135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108920 0x7f7fec10ad40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009ca0 tx=0x7f7fe402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 -- 172.21.15.19:0/2159901135 shutdown_connections 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 --2- 172.21.15.19:0/2159901135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108920 0x7f7fec10ad40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 --2- 172.21.15.19:0/2159901135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fec103c20 0x7f7fec104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 --2- 172.21.15.19:0/2159901135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fec1029e0 0x7f7fec1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 -- 172.21.15.19:0/2159901135 >> 172.21.15.19:0/2159901135 conn(0x7f7fec0fdbd0 msgr2=0x7f7fec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 -- 172.21.15.19:0/2159901135 shutdown_connections 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.733+0000 7f7ff2338640 1 -- 172.21.15.19:0/2159901135 wait complete. 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 Processor -- start 2024-04-25T10:53:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 -- start start 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fec1029e0 0x7f7fec06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec103c20 0x7f7fec1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fec108920 0x7f7fec1a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10e560 con 0x7f7fec103c20 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10e3e0 con 0x7f7fec1029e0 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff2338640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10e6e0 con 0x7f7fec108920 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7feb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec103c20 0x7f7fec1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7feb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec103c20 0x7f7fec1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52080/0 (socket says 172.21.15.19:52080) 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7feb7fe640 1 -- 172.21.15.19:0/3670982462 learned_addr learned my addr 172.21.15.19:0/3670982462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7febfff640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fec1029e0 0x7f7fec06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.734+0000 7f7ff08d5640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fec108920 0x7f7fec1a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7feb7fe640 1 -- 172.21.15.19:0/3670982462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fec108920 msgr2=0x7f7fec1a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7feb7fe640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fec108920 0x7f7fec1a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7feb7fe640 1 -- 172.21.15.19:0/3670982462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fec1029e0 msgr2=0x7f7fec06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7feb7fe640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fec1029e0 0x7f7fec06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7feb7fe640 1 -- 172.21.15.19:0/3670982462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec1a4270 con 0x7f7fec103c20 2024-04-25T10:53:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7ff08d5640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fec108920 0x7f7fec1a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7feb7fe640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec103c20 0x7f7fec1a16e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000b4a0 tx=0x7f7fe000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7fe97fa640 1 -- 172.21.15.19:0/3670982462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0014070 con 0x7f7fec103c20 2024-04-25T10:53:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7fe97fa640 1 -- 172.21.15.19:0/3670982462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe0010ac0 con 0x7f7fec103c20 2024-04-25T10:53:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7fe97fa640 1 -- 172.21.15.19:0/3670982462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0017c30 con 0x7f7fec103c20 2024-04-25T10:53:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec1a4560 con 0x7f7fec103c20 2024-04-25T10:53:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.735+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a4b30 con 0x7f7fec103c20 2024-04-25T10:53:55.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.736+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec107730 con 0x7f7fec103c20 2024-04-25T10:53:55.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.737+0000 7f7fe97fa640 1 -- 172.21.15.19:0/3670982462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f7fe0017070 con 0x7f7fec103c20 2024-04-25T10:53:55.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.738+0000 7f7fe97fa640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7fd40afc00 0x7f7fd40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:55.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.738+0000 7f7febfff640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7fd40afc00 0x7f7fd40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:55.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.738+0000 7f7fe97fa640 1 -- 172.21.15.19:0/3670982462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f7fe00de440 con 0x7f7fec103c20 2024-04-25T10:53:55.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.739+0000 7f7febfff640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7fd40afc00 0x7f7fd40b2110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7fec1035e0 tx=0x7f7fdc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.742+0000 7f7fe97fa640 1 -- 172.21.15.19:0/3670982462 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fe00a6f80 con 0x7f7fec103c20 2024-04-25T10:53:55.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:55 smithi169 ceph-mon[33049]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:55.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:55 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/921660237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:55 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/258028762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:55 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1321862678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:53:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.927+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7fec06c2a0 con 0x7f7fec103c20 2024-04-25T10:53:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.928+0000 7f7fe97fa640 1 -- 172.21.15.19:0/3670982462 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f7fec06c2a0 con 0x7f7fec103c20 2024-04-25T10:53:55.929 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7fd40afc00 msgr2=0x7f7fd40b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7fd40afc00 0x7f7fd40b2110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7fec1035e0 tx=0x7f7fdc00a2d0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec103c20 msgr2=0x7f7fec1a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec103c20 0x7f7fec1a16e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000b4a0 tx=0x7f7fe000b960 comp rx=0 tx=0).stop 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 shutdown_connections 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fec108920 0x7f7fec1a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f7fd40afc00 0x7f7fd40b2110 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec103c20 0x7f7fec1a16e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 --2- 172.21.15.19:0/3670982462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fec1029e0 0x7f7fec06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 >> 172.21.15.19:0/3670982462 conn(0x7f7fec0fdbd0 msgr2=0x7f7fec104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:55.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 shutdown_connections 2024-04-25T10:53:55.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:55.930+0000 7f7ff2338640 1 -- 172.21.15.19:0/3670982462 wait complete. 2024-04-25T10:53:55.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 -- 172.21.15.19:0/3142939578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 msgr2=0x7f2df0105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 --2- 172.21.15.19:0/3142939578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df0105140 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009900 tx=0x7f2de002a410 comp rx=0 tx=0).stop 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 -- 172.21.15.19:0/3142939578 shutdown_connections 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 --2- 172.21.15.19:0/3142939578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109a30 0x7f2df010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 --2- 172.21.15.19:0/3142939578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df0105140 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 --2- 172.21.15.19:0/3142939578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df0103af0 0x7f2df01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 -- 172.21.15.19:0/3142939578 >> 172.21.15.19:0/3142939578 conn(0x7f2df00fece0 msgr2=0x7f2df0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 -- 172.21.15.19:0/3142939578 shutdown_connections 2024-04-25T10:53:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.056+0000 7f2df7b64640 1 -- 172.21.15.19:0/3142939578 wait complete. 2024-04-25T10:53:56.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 Processor -- start 2024-04-25T10:53:56.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 -- start start 2024-04-25T10:53:56.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0103af0 0x7f2df00778f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df0104d30 0x7f2df0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109a30 0x7f2df0076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f610 con 0x7f2df0103af0 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f490 con 0x7f2df0109a30 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df7b64640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f790 con 0x7f2df0104d30 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0103af0 0x7f2df00778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0103af0 0x7f2df00778f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52088/0 (socket says 172.21.15.19:52088) 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 -- 172.21.15.19:0/2862424728 learned_addr learned my addr 172.21.15.19:0/2862424728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df6101640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109a30 0x7f2df0076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 -- 172.21.15.19:0/2862424728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df0104d30 msgr2=0x7f2df0076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df0104d30 0x7f2df0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 -- 172.21.15.19:0/2862424728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109a30 msgr2=0x7f2df0076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109a30 0x7f2df0076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 -- 172.21.15.19:0/2862424728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df0076c70 con 0x7f2df0103af0 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df6101640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109a30 0x7f2df0076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.057+0000 7f2df5900640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0103af0 0x7f2df00778f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2dec005d00 tx=0x7f2dec00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.058+0000 7f2de6ffd640 1 -- 172.21.15.19:0/2862424728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec017070 con 0x7f2df0103af0 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.058+0000 7f2de6ffd640 1 -- 172.21.15.19:0/2862424728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dec009c30 con 0x7f2df0103af0 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.058+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a4390 con 0x7f2df0103af0 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.058+0000 7f2de6ffd640 1 -- 172.21.15.19:0/2862424728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00f620 con 0x7f2df0103af0 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.058+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a47a0 con 0x7f2df0103af0 2024-04-25T10:53:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.059+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df006a160 con 0x7f2df0103af0 2024-04-25T10:53:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.060+0000 7f2de6ffd640 1 -- 172.21.15.19:0/2862424728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2dec01a070 con 0x7f2df0103af0 2024-04-25T10:53:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.061+0000 7f2de6ffd640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ddc0afb80 0x7f2ddc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.061+0000 7f2df50ff640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ddc0afb80 0x7f2ddc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.061+0000 7f2de6ffd640 1 -- 172.21.15.19:0/2862424728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f2dec0de8e0 con 0x7f2df0103af0 2024-04-25T10:53:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.061+0000 7f2df50ff640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ddc0afb80 0x7f2ddc0b2090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2de0031500 tx=0x7f2de0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.066+0000 7f2de6ffd640 1 -- 172.21.15.19:0/2862424728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2dec009de0 con 0x7f2df0103af0 2024-04-25T10:53:56.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.253+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2df002db50 con 0x7f2df0103af0 2024-04-25T10:53:56.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.253+0000 7f2de6ffd640 1 -- 172.21.15.19:0/2862424728 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f2dec0a7310 con 0x7f2df0103af0 2024-04-25T10:53:56.254 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:53:56.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ddc0afb80 msgr2=0x7f2ddc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ddc0afb80 0x7f2ddc0b2090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2de0031500 tx=0x7f2de0031400 comp rx=0 tx=0).stop 2024-04-25T10:53:56.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0103af0 msgr2=0x7f2df00778f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0103af0 0x7f2df00778f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2dec005d00 tx=0x7f2dec00dde0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 shutdown_connections 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109a30 0x7f2df0076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df0104d30 0x7f2df0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2ddc0afb80 0x7f2ddc0b2090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 --2- 172.21.15.19:0/2862424728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0103af0 0x7f2df00778f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.255+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 >> 172.21.15.19:0/2862424728 conn(0x7f2df00fece0 msgr2=0x7f2df0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.256+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 shutdown_connections 2024-04-25T10:53:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.256+0000 7f2df7b64640 1 -- 172.21.15.19:0/2862424728 wait complete. 2024-04-25T10:53:56.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:53:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.382+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/3920712077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41098b0 msgr2=0x7fc0c410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.382+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/3920712077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41098b0 0x7fc0c410bcd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009ca0 tx=0x7fc0bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.382+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/3920712077 shutdown_connections 2024-04-25T10:53:56.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.382+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/3920712077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41098b0 0x7fc0c410bcd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.382+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/3920712077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0c4106f50 0x7fc0c4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.382+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/3920712077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c41045f0 0x7fc0c4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.382+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/3920712077 >> 172.21.15.19:0/3920712077 conn(0x7fc0c40fdbf0 msgr2=0x7fc0c41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:56.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/3920712077 shutdown_connections 2024-04-25T10:53:56.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/3920712077 wait complete. 2024-04-25T10:53:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 Processor -- start 2024-04-25T10:53:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 -- start start 2024-04-25T10:53:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c41045f0 0x7fc0c419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4106f50 0x7fc0c419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0c41098b0 0x7fc0c41a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c410f630 con 0x7fc0c4106f50 2024-04-25T10:53:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c410f4b0 con 0x7fc0c41098b0 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.383+0000 7fc0cb4e2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c410f7b0 con 0x7fc0c41045f0 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4106f50 0x7fc0c419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4106f50 0x7fc0c419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52108/0 (socket says 172.21.15.19:52108) 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 -- 172.21.15.19:0/2476309860 learned_addr learned my addr 172.21.15.19:0/2476309860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c927e640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c41045f0 0x7fc0c419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 -- 172.21.15.19:0/2476309860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c41045f0 msgr2=0x7fc0c419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c41045f0 0x7fc0c419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 -- 172.21.15.19:0/2476309860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0c41098b0 msgr2=0x7fc0c41a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0c41098b0 0x7fc0c41a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 -- 172.21.15.19:0/2476309860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c41a1d40 con 0x7fc0c4106f50 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c927e640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c41045f0 0x7fc0c419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0c8a7d640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4106f50 0x7fc0c419a740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400d950 tx=0x7fc0b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0ba7fc640 1 -- 172.21.15.19:0/2476309860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4017030 con 0x7fc0c4106f50 2024-04-25T10:53:56.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0ba7fc640 1 -- 172.21.15.19:0/2476309860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b4004020 con 0x7fc0c4106f50 2024-04-25T10:53:56.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c41a1fd0 con 0x7fc0c4106f50 2024-04-25T10:53:56.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0ba7fc640 1 -- 172.21.15.19:0/2476309860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40047a0 con 0x7fc0c4106f50 2024-04-25T10:53:56.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.384+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c41a24c0 con 0x7fc0c4106f50 2024-04-25T10:53:56.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.385+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c40ff710 con 0x7fc0c4106f50 2024-04-25T10:53:56.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.387+0000 7fc0ba7fc640 1 -- 172.21.15.19:0/2476309860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc0b4009a30 con 0x7fc0c4106f50 2024-04-25T10:53:56.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.387+0000 7fc0ba7fc640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc0b00afb80 0x7fc0b00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.388+0000 7fc0c927e640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc0b00afb80 0x7fc0b00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.388+0000 7fc0ba7fc640 1 -- 172.21.15.19:0/2476309860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fc0b40def40 con 0x7fc0c4106f50 2024-04-25T10:53:56.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.388+0000 7fc0c927e640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc0b00afb80 0x7fc0b00b2090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40757f0 tx=0x7fc0c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:56.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.391+0000 7fc0ba7fc640 1 -- 172.21.15.19:0/2476309860 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0b40a7970 con 0x7fc0c4106f50 2024-04-25T10:53:56.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:56 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/903143069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:56.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:56 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2616721103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:56.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:56 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3670982462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:53:56.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:56 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2862424728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:53:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.578+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0c402dce0 con 0x7fc0c4106f50 2024-04-25T10:53:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.579+0000 7fc0ba7fc640 1 -- 172.21.15.19:0/2476309860 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7fc0b40a7270 con 0x7fc0c4106f50 2024-04-25T10:53:56.579 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:53:56.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.580+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc0b00afb80 msgr2=0x7fc0b00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.580+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc0b00afb80 0x7fc0b00b2090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40757f0 tx=0x7fc0c0005f90 comp rx=0 tx=0).stop 2024-04-25T10:53:56.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.580+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4106f50 msgr2=0x7fc0c419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.580+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4106f50 0x7fc0c419a740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400d950 tx=0x7fc0b400de10 comp rx=0 tx=0).stop 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 shutdown_connections 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0c41098b0 0x7fc0c41a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fc0b00afb80 0x7fc0b00b2090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4106f50 0x7fc0c419a740 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 --2- 172.21.15.19:0/2476309860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c41045f0 0x7fc0c419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 >> 172.21.15.19:0/2476309860 conn(0x7fc0c40fdbf0 msgr2=0x7fc0c4105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 shutdown_connections 2024-04-25T10:53:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.581+0000 7fc0cb4e2640 1 -- 172.21.15.19:0/2476309860 wait complete. 2024-04-25T10:53:56.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:56 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/903143069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:56.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:56 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2616721103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:56.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:56 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3670982462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:53:56.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:56 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2862424728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:53:56.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:53:56.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.703+0000 7feac1c92640 1 -- 172.21.15.19:0/1676488544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 msgr2=0x7feabc105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.703+0000 7feac1c92640 1 --2- 172.21.15.19:0/1676488544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc105150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feab0009900 tx=0x7feab002a410 comp rx=0 tx=0).stop 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.703+0000 7feac1c92640 1 -- 172.21.15.19:0/1676488544 shutdown_connections 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.703+0000 7feac1c92640 1 --2- 172.21.15.19:0/1676488544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc109a40 0x7feabc10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.703+0000 7feac1c92640 1 --2- 172.21.15.19:0/1676488544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc105150 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.703+0000 7feac1c92640 1 --2- 172.21.15.19:0/1676488544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feabc103b00 0x7feabc104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.703+0000 7feac1c92640 1 -- 172.21.15.19:0/1676488544 >> 172.21.15.19:0/1676488544 conn(0x7feabc0fed10 msgr2=0x7feabc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.704+0000 7feac1c92640 1 -- 172.21.15.19:0/1676488544 shutdown_connections 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.704+0000 7feac1c92640 1 -- 172.21.15.19:0/1676488544 wait complete. 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.704+0000 7feac1c92640 1 Processor -- start 2024-04-25T10:53:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.704+0000 7feac1c92640 1 -- start start 2024-04-25T10:53:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feac1c92640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feabc103b00 0x7feabc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feac1c92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feac1c92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc109a40 0x7feabc1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feac1c92640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10f620 con 0x7feabc104d40 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feac1c92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10f4a0 con 0x7feabc109a40 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feac1c92640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10f7a0 con 0x7feabc103b00 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52134/0 (socket says 172.21.15.19:52134) 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 -- 172.21.15.19:0/2530994535 learned_addr learned my addr 172.21.15.19:0/2530994535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabb7fe640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feabc103b00 0x7feabc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabbfff640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc109a40 0x7feabc1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 -- 172.21.15.19:0/2530994535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feabc103b00 msgr2=0x7feabc19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feabc103b00 0x7feabc19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 -- 172.21.15.19:0/2530994535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc109a40 msgr2=0x7feabc1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc109a40 0x7feabc1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 -- 172.21.15.19:0/2530994535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc1a1d60 con 0x7feabc104d40 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabbfff640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc109a40 0x7feabc1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feabaffd640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc19a760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feab0006010 tx=0x7feab0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feab8ff9640 1 -- 172.21.15.19:0/2530994535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab003f030 con 0x7feabc104d40 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc1a1ff0 con 0x7feabc104d40 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feab8ff9640 1 -- 172.21.15.19:0/2530994535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab0003aa0 con 0x7feabc104d40 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.705+0000 7feab8ff9640 1 -- 172.21.15.19:0/2530994535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0035070 con 0x7feabc104d40 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.706+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a2400 con 0x7feabc104d40 2024-04-25T10:53:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.706+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc108850 con 0x7feabc104d40 2024-04-25T10:53:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.708+0000 7feab8ff9640 1 -- 172.21.15.19:0/2530994535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7feab0042030 con 0x7feabc104d40 2024-04-25T10:53:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.709+0000 7feab8ff9640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feaa40afb80 0x7feaa40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.709+0000 7feabb7fe640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feaa40afb80 0x7feaa40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.709+0000 7feab8ff9640 1 -- 172.21.15.19:0/2530994535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7feab00ffa20 con 0x7feabc104d40 2024-04-25T10:53:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.710+0000 7feabb7fe640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feaa40afb80 0x7feaa40b2090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feabc104700 tx=0x7feaac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.713+0000 7feab8ff9640 1 -- 172.21.15.19:0/2530994535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feab00c84b0 con 0x7feabc104d40 2024-04-25T10:53:56.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:56 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/903143069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:56.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:56 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2616721103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:56.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:56 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3670982462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:53:56.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:56 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2862424728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:53:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.943+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feabc02db50 con 0x7feabc104d40 2024-04-25T10:53:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.944+0000 7feab8ff9640 1 -- 172.21.15.19:0/2530994535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7feab00c7db0 con 0x7feabc104d40 2024-04-25T10:53:56.945 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:56.945 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:53:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feaa40afb80 msgr2=0x7feaa40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feaa40afb80 0x7feaa40b2090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feabc104700 tx=0x7feaac005f90 comp rx=0 tx=0).stop 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 msgr2=0x7feabc19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc19a760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feab0006010 tx=0x7feab0031a50 comp rx=0 tx=0).stop 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 shutdown_connections 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc109a40 0x7feabc1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7feaa40afb80 0x7feaa40b2090 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feabc104d40 0x7feabc19a760 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 --2- 172.21.15.19:0/2530994535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feabc103b00 0x7feabc19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 >> 172.21.15.19:0/2530994535 conn(0x7feabc0fed10 msgr2=0x7feabc105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:56.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 shutdown_connections 2024-04-25T10:53:56.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:56.947+0000 7feac1c92640 1 -- 172.21.15.19:0/2530994535 wait complete. 2024-04-25T10:53:56.959 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:53:57.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 -- 172.21.15.19:0/2717986048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 msgr2=0x7f0ae4106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 --2- 172.21.15.19:0/2717986048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae4106690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009900 tx=0x7f0ad402a410 comp rx=0 tx=0).stop 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 -- 172.21.15.19:0/2717986048 shutdown_connections 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 --2- 172.21.15.19:0/2717986048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae4109770 0x7f0ae410bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 --2- 172.21.15.19:0/2717986048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae4106c60 0x7f0ae41090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 --2- 172.21.15.19:0/2717986048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae4106690 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 -- 172.21.15.19:0/2717986048 >> 172.21.15.19:0/2717986048 conn(0x7f0ae40fdbd0 msgr2=0x7f0ae4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 -- 172.21.15.19:0/2717986048 shutdown_connections 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.071+0000 7f0ae9251640 1 -- 172.21.15.19:0/2717986048 wait complete. 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 Processor -- start 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 -- start start 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae4106c60 0x7f0ae419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae4109770 0x7f0ae41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae410f5e0 con 0x7f0ae4104270 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae410f460 con 0x7f0ae4106c60 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae9251640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae410f760 con 0x7f0ae4109770 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52156/0 (socket says 172.21.15.19:52156) 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.072+0000 7f0ae2d9d640 1 -- 172.21.15.19:0/400645758 learned_addr learned my addr 172.21.15.19:0/400645758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae359e640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae4109770 0x7f0ae41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0adbfff640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae4106c60 0x7f0ae419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae2d9d640 1 -- 172.21.15.19:0/400645758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae4109770 msgr2=0x7f0ae41a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae2d9d640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae4109770 0x7f0ae41a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae2d9d640 1 -- 172.21.15.19:0/400645758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae4106c60 msgr2=0x7f0ae419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae2d9d640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae4106c60 0x7f0ae419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae2d9d640 1 -- 172.21.15.19:0/400645758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae41a1e00 con 0x7f0ae4104270 2024-04-25T10:53:57.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae2d9d640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae419a2c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4006010 tx=0x7f0ad4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae0d99640 1 -- 172.21.15.19:0/400645758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad403f030 con 0x7f0ae4104270 2024-04-25T10:53:57.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae41a2090 con 0x7f0ae4104270 2024-04-25T10:53:57.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae0d99640 1 -- 172.21.15.19:0/400645758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad4003aa0 con 0x7f0ae4104270 2024-04-25T10:53:57.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae0d99640 1 -- 172.21.15.19:0/400645758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4035070 con 0x7f0ae4104270 2024-04-25T10:53:57.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.073+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae41a2580 con 0x7f0ae4104270 2024-04-25T10:53:57.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.074+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae40ff6f0 con 0x7f0ae4104270 2024-04-25T10:53:57.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.076+0000 7f0ae0d99640 1 -- 172.21.15.19:0/400645758 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0ad4042030 con 0x7f0ae4104270 2024-04-25T10:53:57.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.077+0000 7f0ae0d99640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0ad00afb80 0x7f0ad00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.077+0000 7f0adbfff640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0ad00afb80 0x7f0ad00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.077+0000 7f0ae0d99640 1 -- 172.21.15.19:0/400645758 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f0ad40ff830 con 0x7f0ae4104270 2024-04-25T10:53:57.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.078+0000 7f0adbfff640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0ad00afb80 0x7f0ad00b2090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0acc005fd0 tx=0x7f0acc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.081+0000 7f0ae0d99640 1 -- 172.21.15.19:0/400645758 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ad40c8260 con 0x7f0ae4104270 2024-04-25T10:53:57.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.267+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ae402dd20 con 0x7f0ae4104270 2024-04-25T10:53:57.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.269+0000 7f0ae0d99640 1 -- 172.21.15.19:0/400645758 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7f0ad40d02e0 con 0x7f0ae4104270 2024-04-25T10:53:57.270 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:57.270 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0ad00afb80 msgr2=0x7f0ad00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0ad00afb80 0x7f0ad00b2090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0acc005fd0 tx=0x7f0acc005850 comp rx=0 tx=0).stop 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 msgr2=0x7f0ae419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae419a2c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4006010 tx=0x7f0ad4031a50 comp rx=0 tx=0).stop 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 shutdown_connections 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae4109770 0x7f0ae41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae4106c60 0x7f0ae419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f0ad00afb80 0x7f0ad00b2090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 --2- 172.21.15.19:0/400645758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ae4104270 0x7f0ae419a2c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 >> 172.21.15.19:0/400645758 conn(0x7f0ae40fdbd0 msgr2=0x7f0ae4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 shutdown_connections 2024-04-25T10:53:57.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.271+0000 7f0ae9251640 1 -- 172.21.15.19:0/400645758 wait complete. 2024-04-25T10:53:57.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 -- 172.21.15.19:0/3528871260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 msgr2=0x7f63f8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 --2- 172.21.15.19:0/3528871260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f8104800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009900 tx=0x7f63ec02a410 comp rx=0 tx=0).stop 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 -- 172.21.15.19:0/3528871260 shutdown_connections 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 --2- 172.21.15.19:0/3528871260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8109a40 0x7f63f810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 --2- 172.21.15.19:0/3528871260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f8104d40 0x7f63f8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 --2- 172.21.15.19:0/3528871260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f8104800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 -- 172.21.15.19:0/3528871260 >> 172.21.15.19:0/3528871260 conn(0x7f63f80fed10 msgr2=0x7f63f81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 -- 172.21.15.19:0/3528871260 shutdown_connections 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.395+0000 7f63feb72640 1 -- 172.21.15.19:0/3528871260 wait complete. 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 Processor -- start 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 -- start start 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f8104d40 0x7f63f819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8109a40 0x7f63f81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63f810f600 con 0x7f63f8103b00 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63f810f480 con 0x7f63f8104d40 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63feb72640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63f810f780 con 0x7f63f8109a40 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63fc90e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63fc90e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52160/0 (socket says 172.21.15.19:52160) 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.396+0000 7f63fc90e640 1 -- 172.21.15.19:0/3811996143 learned_addr learned my addr 172.21.15.19:0/3811996143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63fd10f640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8109a40 0x7f63f81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63f7fff640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f8104d40 0x7f63f819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63fc90e640 1 -- 172.21.15.19:0/3811996143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8109a40 msgr2=0x7f63f81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63fc90e640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8109a40 0x7f63f81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63fc90e640 1 -- 172.21.15.19:0/3811996143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f8104d40 msgr2=0x7f63f819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63fc90e640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f8104d40 0x7f63f819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63fc90e640 1 -- 172.21.15.19:0/3811996143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f81a1d60 con 0x7f63f8103b00 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63fc90e640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f819a220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006010 tx=0x7f63ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63f5ffb640 1 -- 172.21.15.19:0/3811996143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec03f030 con 0x7f63f8103b00 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63f5ffb640 1 -- 172.21.15.19:0/3811996143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ec031910 con 0x7f63f8103b00 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f81a1ff0 con 0x7f63f8103b00 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63f5ffb640 1 -- 172.21.15.19:0/3811996143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec035070 con 0x7f63f8103b00 2024-04-25T10:53:57.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.397+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f81a2450 con 0x7f63f8103b00 2024-04-25T10:53:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.398+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f8108850 con 0x7f63f8103b00 2024-04-25T10:53:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.400+0000 7f63f5ffb640 1 -- 172.21.15.19:0/3811996143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f63ec042030 con 0x7f63f8103b00 2024-04-25T10:53:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.400+0000 7f63f5ffb640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63e80afc00 0x7f63e80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.400+0000 7f63f7fff640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63e80afc00 0x7f63e80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.400+0000 7f63f5ffb640 1 -- 172.21.15.19:0/3811996143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f63ec0ff140 con 0x7f63f8103b00 2024-04-25T10:53:57.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.401+0000 7f63f7fff640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63e80afc00 0x7f63e80b2110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63f819b2c0 tx=0x7f63e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.405+0000 7f63f5ffb640 1 -- 172.21.15.19:0/3811996143 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63ec0c7bd0 con 0x7f63f8103b00 2024-04-25T10:53:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:57 smithi112 ceph-mon[32799]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:57 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2476309860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:53:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:57 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2530994535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:57 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/400645758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:57 smithi019 ceph-mon[29209]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:57 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2476309860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:53:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:57 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2530994535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:57 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/400645758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.636+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63f802dd70 con 0x7f63f8103b00 2024-04-25T10:53:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.637+0000 7f63f5ffb640 1 -- 172.21.15.19:0/3811996143 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f63ec0c74d0 con 0x7f63f8103b00 2024-04-25T10:53:57.637 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:53:57.637 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:53:57.638 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63e80afc00 msgr2=0x7f63e80b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63e80afc00 0x7f63e80b2110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63f819b2c0 tx=0x7f63e4005f90 comp rx=0 tx=0).stop 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 msgr2=0x7f63f819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f819a220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006010 tx=0x7f63ec004270 comp rx=0 tx=0).stop 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 shutdown_connections 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8109a40 0x7f63f81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f8104d40 0x7f63f819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63e80afc00 0x7f63e80b2110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 --2- 172.21.15.19:0/3811996143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f8103b00 0x7f63f819a220 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 >> 172.21.15.19:0/3811996143 conn(0x7f63f80fed10 msgr2=0x7f63f8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 shutdown_connections 2024-04-25T10:53:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.639+0000 7f63feb72640 1 -- 172.21.15.19:0/3811996143 wait complete. 2024-04-25T10:53:57.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:52:32.071033+0000', 'modified': '2024-04-25T10:52:36.113733+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6839/386384439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 386384439}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 386384439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/850740995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 850740995}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 850740995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24533, 'qdb_cluster': [24533]} 2024-04-25T10:53:57.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:53:57.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:53:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:57 smithi169 ceph-mon[33049]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:53:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:57 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2476309860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:53:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:57 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2530994535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:57 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/400645758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 -- 172.21.15.19:0/2474132017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a80109890 msgr2=0x7f8a8010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 --2- 172.21.15.19:0/2474132017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a80109890 0x7f8a8010bcb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009ca0 tx=0x7f8a7802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 -- 172.21.15.19:0/2474132017 shutdown_connections 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 --2- 172.21.15.19:0/2474132017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a80109890 0x7f8a8010bcb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 --2- 172.21.15.19:0/2474132017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a80106f30 0x7f8a80109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 --2- 172.21.15.19:0/2474132017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a801045d0 0x7f8a801069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 -- 172.21.15.19:0/2474132017 >> 172.21.15.19:0/2474132017 conn(0x7f8a800fdbd0 msgr2=0x7f8a80100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 -- 172.21.15.19:0/2474132017 shutdown_connections 2024-04-25T10:53:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.763+0000 7f8a8693c640 1 -- 172.21.15.19:0/2474132017 wait complete. 2024-04-25T10:53:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 Processor -- start 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 -- start start 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a801045d0 0x7f8a8019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a80106f30 0x7f8a8019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a80109890 0x7f8a801a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f5f0 con 0x7f8a801045d0 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f470 con 0x7f8a80106f30 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a8693c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f770 con 0x7f8a80109890 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a801045d0 0x7f8a8019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a801045d0 0x7f8a8019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52170/0 (socket says 172.21.15.19:52170) 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a7ffff640 1 -- 172.21.15.19:0/3273307527 learned_addr learned my addr 172.21.15.19:0/3273307527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.764+0000 7f8a84ed9640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a80109890 0x7f8a801a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a80106f30 0x7f8a8019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7ffff640 1 -- 172.21.15.19:0/3273307527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a80109890 msgr2=0x7f8a801a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a80109890 0x7f8a801a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7ffff640 1 -- 172.21.15.19:0/3273307527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a80106f30 msgr2=0x7f8a8019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a80106f30 0x7f8a8019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7ffff640 1 -- 172.21.15.19:0/3273307527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a801a1f80 con 0x7f8a801045d0 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a80106f30 0x7f8a8019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a801045d0 0x7f8a8019a4f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000e9c0 tx=0x7f8a7000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7d7fa640 1 -- 172.21.15.19:0/3273307527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000cd80 con 0x7f8a801045d0 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7d7fa640 1 -- 172.21.15.19:0/3273307527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a700054d0 con 0x7f8a801045d0 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a801a2270 con 0x7f8a801045d0 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a7d7fa640 1 -- 172.21.15.19:0/3273307527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70010430 con 0x7f8a801045d0 2024-04-25T10:53:57.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.765+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801a27c0 con 0x7f8a801045d0 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.768+0000 7f8a8d7ea740 1 Processor -- start 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.768+0000 7f8a8d7ea740 1 -- start start 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7d7fa640 1 -- 172.21.15.19:0/3273307527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8a70004fd0 con 0x7f8a801045d0 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51e6590 0x55e2a55af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2a550d740 con 0x55e2a51de3b0 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e2a54a93a0 con 0x55e2a51e6590 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e2a5509700 con 0x55e2a5207270 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55a88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52178/0 (socket says 172.21.15.19:52178) 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 -- 172.21.15.19:0/2898627711 learned_addr learned my addr 172.21.15.19:0/2898627711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51e6590 0x55e2a55af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a84ed9640 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 -- 172.21.15.19:0/2898627711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 msgr2=0x55e2a55a8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55a8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 -- 172.21.15.19:0/2898627711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51e6590 msgr2=0x55e2a55af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51e6590 0x55e2a55af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 -- 172.21.15.19:0/2898627711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a54c5750 con 0x55e2a51de3b0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7d7fa640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a680afb80 0x7f8a680b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.769+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55a88f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a700dcb40 tx=0x7f8a700e73d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe6e3b8d7483bff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.770+0000 7f8a84ed9640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a680afb80 0x7f8a680b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.770+0000 7f8a667fc640 1 -- 172.21.15.19:0/2898627711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700a3490 con 0x55e2a51de3b0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.770+0000 7f8a7d7fa640 1 -- 172.21.15.19:0/3273307527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f8a700df240 con 0x7f8a801045d0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.770+0000 7f8a667fc640 1 -- 172.21.15.19:0/2898627711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a700046c0 con 0x55e2a51de3b0 2024-04-25T10:53:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.770+0000 7f8a667fc640 1 -- 172.21.15.19:0/2898627711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700a6d40 con 0x55e2a51de3b0 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2898627711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 msgr2=0x55e2a55a88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a84ed9640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a680afb80 0x7f8a680b2090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55e2a55a9540 tx=0x7f8a7802f3a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55a88f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a700dcb40 tx=0x7f8a700e73d0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2898627711 shutdown_connections 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51e6590 0x55e2a55af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55a88f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2898627711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2898627711 >> 172.21.15.19:0/2898627711 conn(0x55e2a51fb3c0 msgr2=0x55e2a55a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2898627711 shutdown_connections 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2898627711 wait complete. 2024-04-25T10:53:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 -- start start 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.771+0000 7f8a8d7ea740 1 Processor -- start 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a51e6590 0x55e2a55b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a5207270 0x55e2a55b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55b0240 con 0x55e2a51de3b0 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e2a5494fe0 con 0x55e2a5207270 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55b03c0 con 0x55e2a51e6590 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 10 client.0 _ll_register_callbacks cb 0x55e2a54ab150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52188/0 (socket says 172.21.15.19:52188) 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 -- 172.21.15.19:0/987747024 learned_addr learned my addr 172.21.15.19:0/987747024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a51e6590 0x55e2a55b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a84ed9640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a5207270 0x55e2a55b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 -- 172.21.15.19:0/987747024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a51e6590 msgr2=0x55e2a55b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a51e6590 0x55e2a55b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 -- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a5207270 msgr2=0x55e2a55b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a5207270 0x55e2a55b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 -- 172.21.15.19:0/987747024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a521f2d0 con 0x55e2a51de3b0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a84ed9640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a5207270 0x55e2a55b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55b22d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8a700058c0 tx=0x7f8a7000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17db5407a166bc70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/987747024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700c6060 con 0x55e2a51de3b0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/987747024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f8a700dce50 con 0x55e2a51de3b0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.772+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2a551b350 con 0x55e2a51de3b0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/987747024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8a700a6f20 con 0x55e2a51de3b0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 10 client.15054 fetch_fsmap learned FSMap version 20 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e2a52e96b0 con 0x55e2a51de3b0 2024-04-25T10:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/987747024 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e2a551b350 con 0x55e2a51de3b0 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/987747024 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8a700affa0 con 0x55e2a51de3b0 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 10 client.15054 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 10 client.15054 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 20 client.15054 populate_metadata read hostname 'smithi019' 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a51c5e40 0x55e2a55bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 4 client.15054 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a55151c0 con 0x55e2a51c5e40 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.773+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a51c5e40 0x55e2a55bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.774+0000 7f8a5f7fe640 10 client.15054 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.774+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a51c5e40 0x55e2a55bfac0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3365e63c0078bfd server_cookie=7c072f9d0772e1d9 in_seq=0 out_seq=0 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/987747024 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2a55151c0 con 0x55e2a51c5e40 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a5f7fe640 10 client.15054 handle_command_reply: tid=0 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 client.15054 shutdown 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 msgr2=0x55e2a55b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55b22d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8a700058c0 tx=0x7f8a7000cf20 comp rx=0 tx=0).stop 2024-04-25T10:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 shutdown_connections 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a51c5e40 0x55e2a55bfac0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a5207270 0x55e2a55b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a51e6590 0x55e2a55b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a5f7fe640 0 client.15054 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/987747024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51de3b0 0x55e2a55b22d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 >> 172.21.15.19:0/987747024 conn(0x55e2a51fb3c0 msgr2=0x55e2a55a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 shutdown_connections 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/987747024 wait complete. 2024-04-25T10:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.775+0000 7f8a8d7ea740 20 client.15054 trim_cache size 0 max 16384 2024-04-25T10:53:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 Processor -- start 2024-04-25T10:53:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 -- start start 2024-04-25T10:53:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55bde00 con 0x55e2a51e6590 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55be4d0 con 0x55e2a51de3b0 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55beba0 con 0x55e2a5207270 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52196/0 (socket says 172.21.15.19:52196) 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/106302181 learned_addr learned my addr 172.21.15.19:0/106302181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/106302181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 msgr2=0x55e2a55b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/106302181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 msgr2=0x55e2a55acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.794+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/106302181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a55b3340 con 0x55e2a51e6590 2024-04-25T10:53:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.795+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55a2a40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8a74000c00 tx=0x7f8a7400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec43ae18edaae2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.795+0000 7f8a65ffb640 1 -- 172.21.15.19:0/106302181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74022070 con 0x55e2a51e6590 2024-04-25T10:53:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.795+0000 7f8a65ffb640 1 -- 172.21.15.19:0/106302181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a7401b910 con 0x55e2a51e6590 2024-04-25T10:53:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.795+0000 7f8a65ffb640 1 -- 172.21.15.19:0/106302181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400f8b0 con 0x55e2a51e6590 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.795+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/106302181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 msgr2=0x55e2a55a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.795+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55a2a40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8a74000c00 tx=0x7f8a7400bf90 comp rx=0 tx=0).stop 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/106302181 shutdown_connections 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55a2a40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/106302181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/106302181 >> 172.21.15.19:0/106302181 conn(0x55e2a51fb3c0 msgr2=0x55e2a55a2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/106302181 shutdown_connections 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/106302181 wait complete. 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- start start 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 Processor -- start 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55bf940 con 0x55e2a51e6590 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55bf270 con 0x55e2a51de3b0 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e2a55c0010 con 0x55e2a5207270 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a8d7ea740 10 client.0 _ll_register_callbacks cb 0x7f8a700c4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.796+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52208/0 (socket says 172.21.15.19:52208) 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/2509305704 learned_addr learned my addr 172.21.15.19:0/2509305704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a84ed9640 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/2509305704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 msgr2=0x55e2a55b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55b8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 msgr2=0x55e2a55b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:53:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 -- 172.21.15.19:0/2509305704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a55b9250 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a7f7fe640 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55b5dc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005890 tx=0x7f8a74011300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159875eac269aebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/2509305704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74022040 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a7401b910 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/2509305704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f8a7401b560 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/2509305704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8a7401b760 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a8d7ea740 10 client.15060 fetch_fsmap learned FSMap version 20 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.797+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e2a56289d0 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/2509305704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a7401b910 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/2509305704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8a74025070 con 0x55e2a51e6590 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a8d7ea740 10 client.15060 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a8d7ea740 10 client.15060 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a8d7ea740 20 client.15060 populate_metadata read hostname 'smithi019' 2024-04-25T10:53:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a519b280 0x55e2a562bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a8d7ea740 4 client.15060 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "status"}) -- 0x55e2a54909a0 con 0x55e2a519b280 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.798+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a519b280 0x55e2a562bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a7ffff640 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a519b280 0x55e2a562bf10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14552513f2964a4a server_cookie=d36de079d7a4baa1 in_seq=0 out_seq=0 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a5f7fe640 10 client.15060 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a5f7fe640 1 -- 172.21.15.19:0/2509305704 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e2a54909a0 con 0x55e2a519b280 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a5f7fe640 10 client.15060 handle_command_reply: tid=0 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 client.15060 shutdown 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 msgr2=0x55e2a55b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55b5dc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005890 tx=0x7f8a74011300 comp rx=0 tx=0).stop 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 shutdown_connections 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e2a519b280 0x55e2a562bf10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e2a5207270 0x55e2a55b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a51e6590 0x55e2a55b5dc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a5f7fe640 0 client.15060 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:53:57.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 --2- 172.21.15.19:0/2509305704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2a51de3b0 0x55e2a55b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 >> 172.21.15.19:0/2509305704 conn(0x55e2a51fb3c0 msgr2=0x55e2a55a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 shutdown_connections 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.799+0000 7f8a8d7ea740 1 -- 172.21.15.19:0/2509305704 wait complete. 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8d7ea740 20 client.15060 trim_cache size 0 max 16384 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9e90b6fe-02f0-11ef-bc93-c7b262605968", 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24533, 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:53:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 85.092025237000001, 2024-04-25T10:53:57.806 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 20, 2024-04-25T10:53:57.806 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 79, 2024-04-25T10:53:57.806 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:53:57.806 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 90.819358621999996 2024-04-25T10:53:57.806 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a680afb80 msgr2=0x7f8a680b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a680afb80 0x7f8a680b2090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55e2a55a9540 tx=0x7f8a7802f3a0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a801045d0 msgr2=0x7f8a8019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a801045d0 0x7f8a8019a4f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000e9c0 tx=0x7f8a7000ee80 comp rx=0 tx=0).stop 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 shutdown_connections 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a80109890 0x7f8a801a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a80106f30 0x7f8a8019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f8a680afb80 0x7f8a680b2090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 --2- 172.21.15.19:0/3273307527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a801045d0 0x7f8a8019a4f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 >> 172.21.15.19:0/3273307527 conn(0x7f8a800fdbd0 msgr2=0x7f8a80105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 shutdown_connections 2024-04-25T10:53:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:57.800+0000 7f8a8693c640 1 -- 172.21.15.19:0/3273307527 wait complete. 2024-04-25T10:53:57.812 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T10:53:57.812 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T10:53:57.823 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:53:57.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T10:53:58.070 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:53:58.461 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:58 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3811996143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:58.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:58 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3811996143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:58.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:58 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3811996143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:53:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:53:59 smithi112 ceph-mon[32799]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:59.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:53:59 smithi019 ceph-mon[29209]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:59.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:53:59 smithi169 ceph-mon[33049]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:53:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 -- 172.21.15.19:0/3015682628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d901030d0 msgr2=0x7f3d901034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/3015682628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d901030d0 0x7f3d901034e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009990 tx=0x7f3d84009ca0 comp rx=0 tx=0).stop 2024-04-25T10:53:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 -- 172.21.15.19:0/3015682628 shutdown_connections 2024-04-25T10:53:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/3015682628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d90104ba0 0x7f3d9010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/3015682628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d901030d0 0x7f3d901034e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:59.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/3015682628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d90104200 0x7f3d90104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:59.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 -- 172.21.15.19:0/3015682628 >> 172.21.15.19:0/3015682628 conn(0x7f3d900fe730 msgr2=0x7f3d90100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:53:59.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 -- 172.21.15.19:0/3015682628 shutdown_connections 2024-04-25T10:53:59.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.853+0000 7f3d94b6d700 1 -- 172.21.15.19:0/3015682628 wait complete. 2024-04-25T10:53:59.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.854+0000 7f3d94b6d700 1 Processor -- start 2024-04-25T10:53:59.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.854+0000 7f3d94b6d700 1 -- start start 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d94b6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d901030d0 0x7f3d90199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d94b6d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d90104200 0x7f3d90199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d94b6d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d90104ba0 0x7f3d901a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d94b6d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010eb20 con 0x7f3d90104ba0 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d94b6d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010e9a0 con 0x7f3d901030d0 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d94b6d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010eca0 con 0x7f3d90104200 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d90104ba0 0x7f3d901a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d90104ba0 0x7f3d901a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52232/0 (socket says 172.21.15.19:52232) 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 -- 172.21.15.19:0/370846957 learned_addr learned my addr 172.21.15.19:0/370846957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8e59c700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d901030d0 0x7f3d90199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8dd9b700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d90104200 0x7f3d90199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 -- 172.21.15.19:0/370846957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d90104200 msgr2=0x7f3d90199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d90104200 0x7f3d90199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 -- 172.21.15.19:0/370846957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d901030d0 msgr2=0x7f3d90199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:53:59.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d901030d0 0x7f3d90199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 -- 172.21.15.19:0/370846957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d901a1230 con 0x7f3d90104ba0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d8ed9d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d90104ba0 0x7f3d901a0bb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3d88007ee0 tx=0x7f3d8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d7f7fe700 1 -- 172.21.15.19:0/370846957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8800ddd0 con 0x7f3d90104ba0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.855+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901a1520 con 0x7f3d90104ba0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.856+0000 7f3d7f7fe700 1 -- 172.21.15.19:0/370846957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d88004020 con 0x7f3d90104ba0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.856+0000 7f3d7f7fe700 1 -- 172.21.15.19:0/370846957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88015660 con 0x7f3d90104ba0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.856+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901a1ab0 con 0x7f3d90104ba0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.857+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d90051190 con 0x7f3d90104ba0 2024-04-25T10:53:59.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.858+0000 7f3d7f7fe700 1 -- 172.21.15.19:0/370846957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f3d880041d0 con 0x7f3d90104ba0 2024-04-25T10:53:59.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.859+0000 7f3d7f7fe700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f3d780afc10 0x7f3d780b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:53:59.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.859+0000 7f3d8e59c700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f3d780afc10 0x7f3d780b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:53:59.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.859+0000 7f3d7f7fe700 1 -- 172.21.15.19:0/370846957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f3d880df0b0 con 0x7f3d90104ba0 2024-04-25T10:53:59.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.859+0000 7f3d8e59c700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f3d780afc10 0x7f3d780b2120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3d840038a0 tx=0x7f3d8401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:53:59.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:53:59.862+0000 7f3d7f7fe700 1 -- 172.21.15.19:0/370846957 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d88010070 con 0x7f3d90104ba0 2024-04-25T10:54:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.121+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3d90063ad0 con 0x7f3d780afc10 2024-04-25T10:54:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.123+0000 7f3d7f7fe700 1 -- 172.21.15.19:0/370846957 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3d90063ad0 con 0x7f3d780afc10 2024-04-25T10:54:00.123 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-04-25T10:54:00.123 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-04-25T10:54:00.124 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-04-25T10:54:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f3d780afc10 msgr2=0x7f3d780b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f3d780afc10 0x7f3d780b2120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3d840038a0 tx=0x7f3d8401a040 comp rx=0 tx=0).stop 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d90104ba0 msgr2=0x7f3d901a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d90104ba0 0x7f3d901a0bb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3d88007ee0 tx=0x7f3d8800d3b0 comp rx=0 tx=0).stop 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 shutdown_connections 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f3d780afc10 0x7f3d780b2120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d90104ba0 0x7f3d901a0bb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d901030d0 0x7f3d90199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 --2- 172.21.15.19:0/370846957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d90104200 0x7f3d90199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 >> 172.21.15.19:0/370846957 conn(0x7f3d900fe730 msgr2=0x7f3d90107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 shutdown_connections 2024-04-25T10:54:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:00.125+0000 7f3d94b6d700 1 -- 172.21.15.19:0/370846957 wait complete. 2024-04-25T10:54:00.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T10:54:00.966 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:54:01.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:01 smithi019 ceph-mon[29209]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:01.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:01 smithi019 ceph-mon[29209]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:01.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:01 smithi019 ceph-mon[29209]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T10:54:01.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:01 smithi112 ceph-mon[32799]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:01.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:01 smithi112 ceph-mon[32799]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:01.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:01 smithi112 ceph-mon[32799]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T10:54:01.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:01 smithi169 ceph-mon[33049]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:01.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:01 smithi169 ceph-mon[33049]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:01.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:01 smithi169 ceph-mon[33049]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 -- 172.21.15.19:0/3640534238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3901030e0 msgr2=0x7ff3901034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 --2- 172.21.15.19:0/3640534238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3901030e0 0x7ff3901034f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff380009990 tx=0x7ff380009ca0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 -- 172.21.15.19:0/3640534238 shutdown_connections 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 --2- 172.21.15.19:0/3640534238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff390104bb0 0x7ff39010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 --2- 172.21.15.19:0/3640534238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3901030e0 0x7ff3901034f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 --2- 172.21.15.19:0/3640534238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff390104210 0x7ff390104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 -- 172.21.15.19:0/3640534238 >> 172.21.15.19:0/3640534238 conn(0x7ff3900fe740 msgr2=0x7ff390100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 -- 172.21.15.19:0/3640534238 shutdown_connections 2024-04-25T10:54:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 -- 172.21.15.19:0/3640534238 wait complete. 2024-04-25T10:54:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.517+0000 7ff396623700 1 Processor -- start 2024-04-25T10:54:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff396623700 1 -- start start 2024-04-25T10:54:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff396623700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3901030e0 0x7ff390076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff396623700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff390104210 0x7ff39019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff396623700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff390104bb0 0x7ff3901a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff396623700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff39010eba0 con 0x7ff390104bb0 2024-04-25T10:54:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff396623700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff39010ea20 con 0x7ff3901030e0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff396623700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff39010ed20 con 0x7ff390104210 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff394bc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff390104bb0 0x7ff3901a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff394bc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff390104bb0 0x7ff3901a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52250/0 (socket says 172.21.15.19:52250) 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff394bc0700 1 -- 172.21.15.19:0/2630528402 learned_addr learned my addr 172.21.15.19:0/2630528402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff38ffff700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3901030e0 0x7ff390076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.518+0000 7ff38f7fe700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff390104210 0x7ff39019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff394bc0700 1 -- 172.21.15.19:0/2630528402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff390104210 msgr2=0x7ff39019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff394bc0700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff390104210 0x7ff39019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff394bc0700 1 -- 172.21.15.19:0/2630528402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3901030e0 msgr2=0x7ff390076340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff394bc0700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3901030e0 0x7ff390076340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff394bc0700 1 -- 172.21.15.19:0/2630528402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3901a23a0 con 0x7ff390104bb0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff394bc0700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff390104bb0 0x7ff3901a1b70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b970 tx=0x7ff38800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff38d7fa700 1 -- 172.21.15.19:0/2630528402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388019070 con 0x7ff390104bb0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff38d7fa700 1 -- 172.21.15.19:0/2630528402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff38800cde0 con 0x7ff390104bb0 2024-04-25T10:54:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3901a2690 con 0x7ff390104bb0 2024-04-25T10:54:02.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff38d7fa700 1 -- 172.21.15.19:0/2630528402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388007370 con 0x7ff390104bb0 2024-04-25T10:54:02.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.519+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3901a2c00 con 0x7ff390104bb0 2024-04-25T10:54:02.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.520+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff390051190 con 0x7ff390104bb0 2024-04-25T10:54:02.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.522+0000 7ff38d7fa700 1 -- 172.21.15.19:0/2630528402 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff38801c070 con 0x7ff390104bb0 2024-04-25T10:54:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.522+0000 7ff38d7fa700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3780afb80 0x7ff3780b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.523+0000 7ff38ffff700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3780afb80 0x7ff3780b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.523+0000 7ff38d7fa700 1 -- 172.21.15.19:0/2630528402 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7ff3880de830 con 0x7ff390104bb0 2024-04-25T10:54:02.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.523+0000 7ff38ffff700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3780afb80 0x7ff3780b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3800038a0 tx=0x7ff38001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:02.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.526+0000 7ff38d7fa700 1 -- 172.21.15.19:0/2630528402 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff388023070 con 0x7ff390104bb0 2024-04-25T10:54:02.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.743+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff390063ad0 con 0x7ff3780afb80 2024-04-25T10:54:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.756+0000 7ff38d7fa700 1 -- 172.21.15.19:0/2630528402 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7ff390063ad0 con 0x7ff3780afb80 2024-04-25T10:54:02.757 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T10:54:02.757 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (98s) 18s ago 98s 83.7M - 19.0.0-3291-g994e2f20 d53c96779a29 85a8cb700f38 2024-04-25T10:54:02.757 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi112 running (96s) 29s ago 95s 95.8M - 19.0.0-3291-g994e2f20 d53c96779a29 902bbc03c6c4 2024-04-25T10:54:02.757 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi169 running (110s) 29s ago 109s 80.6M - 19.0.0-3291-g994e2f20 d53c96779a29 14a5ea9b2937 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (118s) 18s ago 117s 86.0M - 19.0.0-3291-g994e2f20 d53c96779a29 9fa8ac13a722 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi112 running (115s) 29s ago 115s 80.6M - 19.0.0-3291-g994e2f20 d53c96779a29 3617a79f9c35 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi169 running (112s) 29s ago 112s 80.4M - 19.0.0-3291-g994e2f20 d53c96779a29 716ab5d21bb9 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (101s) 18s ago 101s 81.3M - 19.0.0-3291-g994e2f20 d53c96779a29 1ca4506a6f3a 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi112 running (107s) 29s ago 106s 81.0M - 19.0.0-3291-g994e2f20 d53c96779a29 ffa8dbdcee34 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi169 running (104s) 29s ago 104s 87.9M - 19.0.0-3291-g994e2f20 d53c96779a29 4dcb3f51c652 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (2m) 18s ago 2m 80.8M - 19.0.0-3291-g994e2f20 d53c96779a29 27ae0acb694b 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi112 running (2m) 29s ago 2m 83.8M - 19.0.0-3291-g994e2f20 d53c96779a29 8683e53191b0 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi169 running (2m) 29s ago 2m 81.7M - 19.0.0-3291-g994e2f20 d53c96779a29 6ffd791b24c6 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (9m) 18s ago 9m 582M - 19.0.0-3291-g994e2f20 d53c96779a29 51e0f592b2ee 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi112 *:8443,8765 running (7m) 29s ago 7m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 9d341b766b7d 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi169 *:8443,8765 running (7m) 29s ago 7m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 43f2139ac85b 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (9m) 18s ago 9m 117M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 ff48db461b11 2024-04-25T10:54:02.758 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi112 running (7m) 29s ago 7m 107M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 2062cf52b06d 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi169 running (7m) 29s ago 7m 105M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 9313eeb8a1be 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 18s ago 7m 115M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 a13218be2fca 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi112 running (6m) 29s ago 6m 113M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 5a0315afdfc7 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi169 running (6m) 29s ago 6m 118M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 01686abd820b 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (5m) 18s ago 5m 121M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 c4f5a22e9354 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi112 running (5m) 29s ago 5m 120M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 6fc74b49eaa1 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi169 running (4m) 29s ago 4m 115M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 97361c3f4773 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (4m) 18s ago 4m 118M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 a030e9c52e3b 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi112 running (4m) 29s ago 4m 114M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 25911ca55c35 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi169 running (3m) 29s ago 3m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 e388a2833af7 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (3m) 18s ago 3m 117M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 6c2ec5f6463f 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi112 running (2m) 29s ago 2m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 58a98328c88f 2024-04-25T10:54:02.759 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi169 running (2m) 29s ago 2m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 021686a53af9 2024-04-25T10:54:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.759+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3780afb80 msgr2=0x7ff3780b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.759+0000 7ff396623700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3780afb80 0x7ff3780b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3800038a0 tx=0x7ff38001a040 comp rx=0 tx=0).stop 2024-04-25T10:54:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.759+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff390104bb0 msgr2=0x7ff3901a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.759+0000 7ff396623700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff390104bb0 0x7ff3901a1b70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b970 tx=0x7ff38800bc80 comp rx=0 tx=0).stop 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 shutdown_connections 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3780afb80 0x7ff3780b2090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff390104bb0 0x7ff3901a1b70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3901030e0 0x7ff390076340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 --2- 172.21.15.19:0/2630528402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff390104210 0x7ff39019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 >> 172.21.15.19:0/2630528402 conn(0x7ff3900fe740 msgr2=0x7ff390107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 shutdown_connections 2024-04-25T10:54:02.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:02.760+0000 7ff396623700 1 -- 172.21.15.19:0/2630528402 wait complete. 2024-04-25T10:54:03.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:03 smithi019 ceph-mon[29209]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:03.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:03 smithi019 ceph-mon[29209]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:03 smithi112 ceph-mon[32799]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:03 smithi112 ceph-mon[32799]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:03.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T10:54:03.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:03 smithi169 ceph-mon[33049]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:03.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:03 smithi169 ceph-mon[33049]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:54:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.421+0000 7f0128324700 1 -- 172.21.15.19:0/1643491258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01201030e0 msgr2=0x7f01201034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.421+0000 7f0128324700 1 --2- 172.21.15.19:0/1643491258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01201030e0 0x7f01201034f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f011c009990 tx=0x7f011c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.421+0000 7f0128324700 1 -- 172.21.15.19:0/1643491258 shutdown_connections 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.421+0000 7f0128324700 1 --2- 172.21.15.19:0/1643491258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0120104bb0 0x7f012010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.421+0000 7f0128324700 1 --2- 172.21.15.19:0/1643491258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01201030e0 0x7f01201034f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.421+0000 7f0128324700 1 --2- 172.21.15.19:0/1643491258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0120104210 0x7f0120104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.421+0000 7f0128324700 1 -- 172.21.15.19:0/1643491258 >> 172.21.15.19:0/1643491258 conn(0x7f01200fe740 msgr2=0x7f0120100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.422+0000 7f0128324700 1 -- 172.21.15.19:0/1643491258 shutdown_connections 2024-04-25T10:54:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.422+0000 7f0128324700 1 -- 172.21.15.19:0/1643491258 wait complete. 2024-04-25T10:54:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.422+0000 7f0128324700 1 Processor -- start 2024-04-25T10:54:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.422+0000 7f0128324700 1 -- start start 2024-04-25T10:54:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f0128324700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01201030e0 0x7f012019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f0128324700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0120104210 0x7f012019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f0128324700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0120104bb0 0x7f01201a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f0128324700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f012010eb30 con 0x7f0120104bb0 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f0128324700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f012010e9b0 con 0x7f01201030e0 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f0128324700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f012010ecb0 con 0x7f0120104210 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0120104bb0 0x7f01201a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0120104bb0 0x7f01201a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34322/0 (socket says 172.21.15.19:34322) 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 -- 172.21.15.19:0/2671903660 learned_addr learned my addr 172.21.15.19:0/2671903660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01260c0700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01201030e0 0x7f012019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01258bf700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0120104210 0x7f012019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 -- 172.21.15.19:0/2671903660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0120104210 msgr2=0x7f012019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0120104210 0x7f012019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 -- 172.21.15.19:0/2671903660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01201030e0 msgr2=0x7f012019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01201030e0 0x7f012019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.423+0000 7f01268c1700 1 -- 172.21.15.19:0/2671903660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01201a2380 con 0x7f0120104bb0 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.424+0000 7f01268c1700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0120104bb0 0x7f01201a1d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f011800ada0 tx=0x7f011800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.424+0000 7f01177fe700 1 -- 172.21.15.19:0/2671903660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118017020 con 0x7f0120104bb0 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.424+0000 7f01177fe700 1 -- 172.21.15.19:0/2671903660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f011800cd80 con 0x7f0120104bb0 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.424+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01201a2670 con 0x7f0120104bb0 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.424+0000 7f01177fe700 1 -- 172.21.15.19:0/2671903660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118005260 con 0x7f0120104bb0 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.424+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201a2b60 con 0x7f0120104bb0 2024-04-25T10:54:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.425+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0120051190 con 0x7f0120104bb0 2024-04-25T10:54:05.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.427+0000 7f01177fe700 1 -- 172.21.15.19:0/2671903660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f011800a090 con 0x7f0120104bb0 2024-04-25T10:54:05.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.428+0000 7f01177fe700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f010c0afb80 0x7f010c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.428+0000 7f01260c0700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f010c0afb80 0x7f010c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.428+0000 7f01177fe700 1 -- 172.21.15.19:0/2671903660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f01180de960 con 0x7f0120104bb0 2024-04-25T10:54:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.428+0000 7f01260c0700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f010c0afb80 0x7f010c0b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f011c0038a0 tx=0x7f011c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.431+0000 7f01177fe700 1 -- 172.21.15.19:0/2671903660 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0118021020 con 0x7f0120104bb0 2024-04-25T10:54:05.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:05 smithi019 ceph-mon[29209]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:54:05.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:05 smithi112 ceph-mon[32799]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:54:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.648+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f01201007e0 con 0x7f010c0afb80 2024-04-25T10:54:05.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.654+0000 7f01177fe700 1 -- 172.21.15.19:0/2671903660 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f01201007e0 con 0x7f010c0afb80 2024-04-25T10:54:05.655 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T10:54:05.655 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 32s ago 2m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T10:54:05.655 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 32s ago 7m smithi019=x;smithi112=y;smithi169=z;count:3 2024-04-25T10:54:05.655 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 32s ago 7m smithi019:172.21.15.19=a;smithi112:172.21.15.112=b;smithi169:172.21.15.169=c;count:3 2024-04-25T10:54:05.655 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 32s ago - 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f010c0afb80 msgr2=0x7f010c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f010c0afb80 0x7f010c0b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f011c0038a0 tx=0x7f011c01a040 comp rx=0 tx=0).stop 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0120104bb0 msgr2=0x7f01201a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0120104bb0 0x7f01201a1d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f011800ada0 tx=0x7f011800c330 comp rx=0 tx=0).stop 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 shutdown_connections 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f010c0afb80 0x7f010c0b2090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0120104bb0 0x7f01201a1d00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01201030e0 0x7f012019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 --2- 172.21.15.19:0/2671903660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0120104210 0x7f012019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 >> 172.21.15.19:0/2671903660 conn(0x7f01200fe740 msgr2=0x7f01201000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 shutdown_connections 2024-04-25T10:54:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:05.656+0000 7f0128324700 1 -- 172.21.15.19:0/2671903660 wait complete. 2024-04-25T10:54:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:05 smithi169 ceph-mon[33049]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:54:06.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:06 smithi019 ceph-mon[29209]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:06.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T10:54:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:06 smithi112 ceph-mon[32799]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:06.579 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:54:06.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:06 smithi169 ceph-mon[33049]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:07.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:07 smithi112 ceph-mon[32799]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:07.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:07 smithi019 ceph-mon[29209]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:07 smithi169 ceph-mon[33049]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 -- 172.21.15.19:0/1155660890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c1030e0 msgr2=0x7f640c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1155660890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c1030e0 0x7f640c1034f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009990 tx=0x7f63fc009ca0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 -- 172.21.15.19:0/1155660890 shutdown_connections 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1155660890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c104bb0 0x7f640c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1155660890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c1030e0 0x7f640c1034f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1155660890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c104210 0x7f640c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 -- 172.21.15.19:0/1155660890 >> 172.21.15.19:0/1155660890 conn(0x7f640c0fe740 msgr2=0x7f640c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 -- 172.21.15.19:0/1155660890 shutdown_connections 2024-04-25T10:54:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 -- 172.21.15.19:0/1155660890 wait complete. 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.224+0000 7f6411fb9700 1 Processor -- start 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f6411fb9700 1 -- start start 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f6411fb9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c1030e0 0x7f640c076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f6411fb9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c104210 0x7f640c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f6411fb9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c104bb0 0x7f640c1a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f6411fb9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10ec20 con 0x7f640c104210 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f6411fb9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10eaa0 con 0x7f640c1030e0 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f6411fb9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10eda0 con 0x7f640c104bb0 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f640affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c104210 0x7f640c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f640affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c104210 0x7f640c1a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34340/0 (socket says 172.21.15.19:34340) 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f640affd700 1 -- 172.21.15.19:0/1198670285 learned_addr learned my addr 172.21.15.19:0/1198670285 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f640b7fe700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c1030e0 0x7f640c076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.225+0000 7f640bfff700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c104bb0 0x7f640c1a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f640affd700 1 -- 172.21.15.19:0/1198670285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c104bb0 msgr2=0x7f640c1a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f640affd700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c104bb0 0x7f640c1a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f640affd700 1 -- 172.21.15.19:0/1198670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c1030e0 msgr2=0x7f640c076320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f640affd700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c1030e0 0x7f640c076320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f640affd700 1 -- 172.21.15.19:0/1198670285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1a46a0 con 0x7f640c104210 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f640affd700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c104210 0x7f640c1a1ae0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f640000b530 tx=0x7f640000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f6408ff9700 1 -- 172.21.15.19:0/1198670285 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400015020 con 0x7f640c104210 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f6408ff9700 1 -- 172.21.15.19:0/1198670285 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6400011ce0 con 0x7f640c104210 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f6408ff9700 1 -- 172.21.15.19:0/1198670285 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400010dd0 con 0x7f640c104210 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1a4990 con 0x7f640c104210 2024-04-25T10:54:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.226+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a4f00 con 0x7f640c104210 2024-04-25T10:54:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.228+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c051190 con 0x7f640c104210 2024-04-25T10:54:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.229+0000 7f6408ff9700 1 -- 172.21.15.19:0/1198670285 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6400018020 con 0x7f640c104210 2024-04-25T10:54:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.230+0000 7f6408ff9700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63f40afb80 0x7f63f40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.230+0000 7f640b7fe700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63f40afb80 0x7f63f40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.230+0000 7f6408ff9700 1 -- 172.21.15.19:0/1198670285 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f64000de8d0 con 0x7f640c104210 2024-04-25T10:54:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.230+0000 7f640b7fe700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63f40afb80 0x7f63f40b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0038a0 tx=0x7f63fc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:08.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.233+0000 7f6408ff9700 1 -- 172.21.15.19:0/1198670285 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f640001f020 con 0x7f640c104210 2024-04-25T10:54:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.448+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f640c063ad0 con 0x7f63f40afb80 2024-04-25T10:54:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.450+0000 7f6408ff9700 1 -- 172.21.15.19:0/1198670285 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f640c063ad0 con 0x7f63f40afb80 2024-04-25T10:54:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-04-25T10:54:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-04-25T10:54:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:smithi112 172.21.15.112 2024-04-25T10:54:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:smithi169 172.21.15.169 2024-04-25T10:54:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-04-25T10:54:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63f40afb80 msgr2=0x7f63f40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63f40afb80 0x7f63f40b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0038a0 tx=0x7f63fc01a040 comp rx=0 tx=0).stop 2024-04-25T10:54:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c104210 msgr2=0x7f640c1a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c104210 0x7f640c1a1ae0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f640000b530 tx=0x7f640000b840 comp rx=0 tx=0).stop 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 shutdown_connections 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63f40afb80 0x7f63f40b2090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c104bb0 0x7f640c1a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c1030e0 0x7f640c076320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 --2- 172.21.15.19:0/1198670285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640c104210 0x7f640c1a1ae0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.452+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 >> 172.21.15.19:0/1198670285 conn(0x7f640c0fe740 msgr2=0x7f640c107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.453+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 shutdown_connections 2024-04-25T10:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:08.453+0000 7f6411fb9700 1 -- 172.21.15.19:0/1198670285 wait complete. 2024-04-25T10:54:09.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T10:54:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:54:09.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:09 smithi112 ceph-mon[32799]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:09.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:09 smithi112 ceph-mon[32799]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:09.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:09 smithi019 ceph-mon[29209]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:09.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:09 smithi019 ceph-mon[29209]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:09.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:09 smithi169 ceph-mon[33049]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:09.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:09 smithi169 ceph-mon[33049]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 -- 172.21.15.19:0/3192374034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 msgr2=0x7f241c104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 --2- 172.21.15.19:0/3192374034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c104620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2410009990 tx=0x7f2410009ca0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 -- 172.21.15.19:0/3192374034 shutdown_connections 2024-04-25T10:54:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 --2- 172.21.15.19:0/3192374034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c104bb0 0x7f241c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 --2- 172.21.15.19:0/3192374034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f241c1030e0 0x7f241c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 --2- 172.21.15.19:0/3192374034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c104620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 -- 172.21.15.19:0/3192374034 >> 172.21.15.19:0/3192374034 conn(0x7f241c0fe740 msgr2=0x7f241c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 -- 172.21.15.19:0/3192374034 shutdown_connections 2024-04-25T10:54:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.258+0000 7f2420942700 1 -- 172.21.15.19:0/3192374034 wait complete. 2024-04-25T10:54:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 Processor -- start 2024-04-25T10:54:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 -- start start 2024-04-25T10:54:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f241c1030e0 0x7f241c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c104bb0 0x7f241c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10eb60 con 0x7f241c104210 2024-04-25T10:54:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10e9e0 con 0x7f241c1030e0 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2420942700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10ece0 con 0x7f241c104bb0 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2419d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2419d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34356/0 (socket says 172.21.15.19:34356) 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.259+0000 7f2419d9b700 1 -- 172.21.15.19:0/415641440 learned_addr learned my addr 172.21.15.19:0/415641440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f241ad9d700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c104bb0 0x7f241c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f241a59c700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f241c1030e0 0x7f241c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2419d9b700 1 -- 172.21.15.19:0/415641440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c104bb0 msgr2=0x7f241c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2419d9b700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c104bb0 0x7f241c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2419d9b700 1 -- 172.21.15.19:0/415641440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f241c1030e0 msgr2=0x7f241c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2419d9b700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f241c1030e0 0x7f241c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2419d9b700 1 -- 172.21.15.19:0/415641440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c1a1170 con 0x7f241c104210 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f241a59c700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f241c1030e0 0x7f241c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2419d9b700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c199be0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2410006010 tx=0x7f24100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f240b7fe700 1 -- 172.21.15.19:0/415641440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241002a070 con 0x7f241c104210 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c0758c0 con 0x7f241c104210 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f240b7fe700 1 -- 172.21.15.19:0/415641440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f241001ac00 con 0x7f241c104210 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f240b7fe700 1 -- 172.21.15.19:0/415641440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24100041f0 con 0x7f241c104210 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.260+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c075da0 con 0x7f241c104210 2024-04-25T10:54:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.261+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c051190 con 0x7f241c104210 2024-04-25T10:54:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.263+0000 7f240b7fe700 1 -- 172.21.15.19:0/415641440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f241002d070 con 0x7f241c104210 2024-04-25T10:54:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.263+0000 7f240b7fe700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24040afb80 0x7f24040b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.263+0000 7f241a59c700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24040afb80 0x7f24040b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.263+0000 7f240b7fe700 1 -- 172.21.15.19:0/415641440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f24100e1e40 con 0x7f241c104210 2024-04-25T10:54:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.264+0000 7f241a59c700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24040afb80 0x7f24040b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f240c005fd0 tx=0x7f240c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.267+0000 7f240b7fe700 1 -- 172.21.15.19:0/415641440 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2410013070 con 0x7f241c104210 2024-04-25T10:54:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.482+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 --> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f241c0760c0 con 0x7f24040afb80 2024-04-25T10:54:11.483 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:11 smithi019 ceph-mon[29209]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.484+0000 7f240b7fe700 1 -- 172.21.15.19:0/415641440 <== mgr.14150 v2:172.21.15.19:6800/1708221429 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f241c0760c0 con 0x7f24040afb80 2024-04-25T10:54:11.485 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T10:54:11.485 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T10:54:11.485 INFO:teuthology.orchestra.run.smithi019.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T10:54:11.485 INFO:teuthology.orchestra.run.smithi019.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24040afb80 msgr2=0x7f24040b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24040afb80 0x7f24040b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f240c005fd0 tx=0x7f240c005e40 comp rx=0 tx=0).stop 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 msgr2=0x7f241c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c199be0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2410006010 tx=0x7f24100047b0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 shutdown_connections 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f24040afb80 0x7f24040b2090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c104bb0 0x7f241c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f241c1030e0 0x7f241c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 --2- 172.21.15.19:0/415641440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f241c104210 0x7f241c199be0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 >> 172.21.15.19:0/415641440 conn(0x7f241c0fe740 msgr2=0x7f241c107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 shutdown_connections 2024-04-25T10:54:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:11.486+0000 7f2420942700 1 -- 172.21.15.19:0/415641440 wait complete. 2024-04-25T10:54:11.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:11 smithi169 ceph-mon[33049]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:11.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:11 smithi112 ceph-mon[32799]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:12.200 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T10:54:12.211 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:54:12.211 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T10:54:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:54:12.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:12 smithi019 ceph-mon[29209]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:12.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:12 smithi169 ceph-mon[33049]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:12.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:12 smithi112 ceph-mon[32799]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:13.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:13 smithi019 ceph-mon[29209]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:13 smithi169 ceph-mon[33049]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:13.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:13 smithi112 ceph-mon[32799]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:54:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 -- 172.21.15.19:0/2478761879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68103100 msgr2=0x7f6c68103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/2478761879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68103100 0x7f6c68103510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009990 tx=0x7f6c58009ca0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 -- 172.21.15.19:0/2478761879 shutdown_connections 2024-04-25T10:54:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/2478761879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c68104bd0 0x7f6c6810b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/2478761879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68103100 0x7f6c68103510 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/2478761879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c68104230 0x7f6c68104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 -- 172.21.15.19:0/2478761879 >> 172.21.15.19:0/2478761879 conn(0x7f6c680fe740 msgr2=0x7f6c68100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 -- 172.21.15.19:0/2478761879 shutdown_connections 2024-04-25T10:54:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.273+0000 7f6c6cd79700 1 -- 172.21.15.19:0/2478761879 wait complete. 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.274+0000 7f6c6cd79700 1 Processor -- start 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.274+0000 7f6c6cd79700 1 -- start start 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c6cd79700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c68103100 0x7f6c6806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c6cd79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c68104230 0x7f6c6819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c6cd79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68104bd0 0x7f6c681a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c6cd79700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6810eb40 con 0x7f6c68104bd0 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c6cd79700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6810e9c0 con 0x7f6c68104230 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c6cd79700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6810ecc0 con 0x7f6c68103100 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68104bd0 0x7f6c681a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68104bd0 0x7f6c681a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43918/0 (socket says 172.21.15.19:43918) 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 -- 172.21.15.19:0/3587388075 learned_addr learned my addr 172.21.15.19:0/3587388075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c65d9b700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c68104230 0x7f6c6819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c6659c700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c68103100 0x7f6c6806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 -- 172.21.15.19:0/3587388075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c68103100 msgr2=0x7f6c6806c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c68103100 0x7f6c6806c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 -- 172.21.15.19:0/3587388075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c68104230 msgr2=0x7f6c6819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c68104230 0x7f6c6819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 -- 172.21.15.19:0/3587388075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c681a22e0 con 0x7f6c68104bd0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.275+0000 7f6c66d9d700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68104bd0 0x7f6c681a1b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ada0 tx=0x7f6c6000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.276+0000 7f6c577fe700 1 -- 172.21.15.19:0/3587388075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60017020 con 0x7f6c68104bd0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.276+0000 7f6c577fe700 1 -- 172.21.15.19:0/3587388075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6000cd80 con 0x7f6c68104bd0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.276+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c681a25d0 con 0x7f6c68104bd0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.276+0000 7f6c577fe700 1 -- 172.21.15.19:0/3587388075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60005260 con 0x7f6c68104bd0 2024-04-25T10:54:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.276+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c681a2ac0 con 0x7f6c68104bd0 2024-04-25T10:54:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.277+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c68051190 con 0x7f6c68104bd0 2024-04-25T10:54:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.278+0000 7f6c577fe700 1 -- 172.21.15.19:0/3587388075 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6c6000a090 con 0x7f6c68104bd0 2024-04-25T10:54:14.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.279+0000 7f6c577fe700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6c500afb80 0x7f6c500b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:14.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.279+0000 7f6c6659c700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6c500afb80 0x7f6c500b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:14.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.279+0000 7f6c577fe700 1 -- 172.21.15.19:0/3587388075 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f6c600de900 con 0x7f6c68104bd0 2024-04-25T10:54:14.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.280+0000 7f6c6659c700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6c500afb80 0x7f6c500b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c580038a0 tx=0x7f6c5801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:14.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.283+0000 7f6c577fe700 1 -- 172.21.15.19:0/3587388075 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c60021020 con 0x7f6c68104bd0 2024-04-25T10:54:14.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.536+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6c6802e510 con 0x7f6c68104bd0 2024-04-25T10:54:14.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.536+0000 7f6c577fe700 1 -- 172.21.15.19:0/3587388075 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3000 (secure 0 0 0) 0x7f6c600a7580 con 0x7f6c68104bd0 2024-04-25T10:54:14.537 INFO:teuthology.orchestra.run.smithi019.stdout:e20 2024-04-25T10:54:14.537 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T10:54:14.537 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:54:14.537 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 20 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T10:52:32.071033+0000 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-25T10:52:36.113733+0000 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-04-25T10:54:14.538 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 1 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:in 0 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24533} 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-04-25T10:54:14.539 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24533 members: 24533 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{0:24533} state up:active seq 3 addr [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{0:24520} state up:standby-replay seq 1 addr [v2:172.21.15.169:6838/386384439,v1:172.21.15.169:6839/386384439] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.19:6834/311381787,v1:172.21.15.19:6835/311381787] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.19:6836/3384911547,v1:172.21.15.19:6837/3384911547] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{-1:14628} state up:standby seq 1 addr [v2:172.21.15.19:6838/154007495,v1:172.21.15.19:6839/154007495] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{-1:14640} state up:standby seq 1 addr [v2:172.21.15.19:6840/427683873,v1:172.21.15.19:6841/427683873] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{-1:24419} state up:standby seq 1 addr [v2:172.21.15.112:6832/1523385162,v1:172.21.15.112:6833/1523385162] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{-1:24448} state up:standby seq 1 addr [v2:172.21.15.169:6832/894672787,v1:172.21.15.169:6833/894672787] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.540 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{-1:24452} state up:standby seq 1 addr [v2:172.21.15.112:6834/392627197,v1:172.21.15.112:6835/392627197] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.541 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{-1:24488} state up:standby seq 1 addr [v2:172.21.15.112:6836/820120755,v1:172.21.15.112:6837/820120755] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.541 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{-1:24493} state up:standby seq 1 addr [v2:172.21.15.169:6834/1662431671,v1:172.21.15.169:6835/1662431671] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.541 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{-1:24505} state up:standby seq 1 addr [v2:172.21.15.169:6836/717205661,v1:172.21.15.169:6837/717205661] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.539+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6c500afb80 msgr2=0x7f6c500b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6c500afb80 0x7f6c500b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c580038a0 tx=0x7f6c5801a040 comp rx=0 tx=0).stop 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68104bd0 msgr2=0x7f6c681a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68104bd0 0x7f6c681a1b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ada0 tx=0x7f6c6000c330 comp rx=0 tx=0).stop 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 shutdown_connections 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f6c500afb80 0x7f6c500b2090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c68104bd0 0x7f6c681a1b40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c68103100 0x7f6c6806c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 --2- 172.21.15.19:0/3587388075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c68104230 0x7f6c6819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 >> 172.21.15.19:0/3587388075 conn(0x7f6c680fe740 msgr2=0x7f6c681074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 shutdown_connections 2024-04-25T10:54:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:14.540+0000 7f6c6cd79700 1 -- 172.21.15.19:0/3587388075 wait complete. 2024-04-25T10:54:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:54:15.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T10:54:15.441 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:54:15.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:15 smithi019 ceph-mon[29209]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T10:54:15.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:15 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3587388075' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T10:54:15.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:15 smithi169 ceph-mon[33049]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T10:54:15.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:15 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3587388075' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T10:54:15.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:15 smithi112 ceph-mon[32799]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T10:54:15.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:15 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3587388075' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T10:54:17.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 -- 172.21.15.19:0/2079726058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388104bf0 msgr2=0x7f638810b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:17.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 --2- 172.21.15.19:0/2079726058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388104bf0 0x7f638810b4a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f638000b0a0 tx=0x7f638000b3b0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 -- 172.21.15.19:0/2079726058 shutdown_connections 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 --2- 172.21.15.19:0/2079726058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388104bf0 0x7f638810b4a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 --2- 172.21.15.19:0/2079726058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6388103120 0x7f6388103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 --2- 172.21.15.19:0/2079726058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6388104250 0x7f6388104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 -- 172.21.15.19:0/2079726058 >> 172.21.15.19:0/2079726058 conn(0x7f63880fe760 msgr2=0x7f6388100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 -- 172.21.15.19:0/2079726058 shutdown_connections 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.026+0000 7f638e86d700 1 -- 172.21.15.19:0/2079726058 wait complete. 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.027+0000 7f638e86d700 1 Processor -- start 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.027+0000 7f638e86d700 1 -- start start 2024-04-25T10:54:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.027+0000 7f638e86d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388103120 0x7f638819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.027+0000 7f638e86d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6388104250 0x7f638819d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.027+0000 7f638e86d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6388104bf0 0x7f63881a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.027+0000 7f638e86d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f638810ecd0 con 0x7f6388103120 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f638e86d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f638810eb50 con 0x7f6388104250 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f638e86d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f638810ee50 con 0x7f6388104bf0 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388103120 0x7f638819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388103120 0x7f638819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43934/0 (socket says 172.21.15.19:43934) 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 -- 172.21.15.19:0/1226753227 learned_addr learned my addr 172.21.15.19:0/1226753227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f63877fe700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6388104250 0x7f638819d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f638ce0a700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6388104bf0 0x7f63881a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 -- 172.21.15.19:0/1226753227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6388104bf0 msgr2=0x7f63881a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6388104bf0 0x7f63881a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 -- 172.21.15.19:0/1226753227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6388104250 msgr2=0x7f638819d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6388104250 0x7f638819d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 -- 172.21.15.19:0/1226753227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63881a4690 con 0x7f6388103120 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f6387fff700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388103120 0x7f638819cb30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f637800b4d0 tx=0x7f637800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.028+0000 7f63857fa700 1 -- 172.21.15.19:0/1226753227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378015020 con 0x7f6388103120 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.029+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63881a4980 con 0x7f6388103120 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.029+0000 7f63857fa700 1 -- 172.21.15.19:0/1226753227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6378011ce0 con 0x7f6388103120 2024-04-25T10:54:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.029+0000 7f63857fa700 1 -- 172.21.15.19:0/1226753227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378010cf0 con 0x7f6388103120 2024-04-25T10:54:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.029+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63881a4f40 con 0x7f6388103120 2024-04-25T10:54:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.030+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6388051190 con 0x7f6388103120 2024-04-25T10:54:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.031+0000 7f63857fa700 1 -- 172.21.15.19:0/1226753227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6378018020 con 0x7f6388103120 2024-04-25T10:54:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.032+0000 7f63857fa700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63700afb80 0x7f63700b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.032+0000 7f63877fe700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63700afb80 0x7f63700b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.032+0000 7f63857fa700 1 -- 172.21.15.19:0/1226753227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f63780de340 con 0x7f6388103120 2024-04-25T10:54:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.032+0000 7f63877fe700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63700afb80 0x7f63700b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f638819dbe0 tx=0x7f637c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.037+0000 7f63857fa700 1 -- 172.21.15.19:0/1226753227 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f637801f020 con 0x7f6388103120 2024-04-25T10:54:17.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.245+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f63881a5170 con 0x7f6388103120 2024-04-25T10:54:17.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.246+0000 7f63857fa700 1 -- 172.21.15.19:0/1226753227 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) ==== 56+0+5282 (secure 0 0 0) 0x7f63780a6f30 con 0x7f6388103120 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 79 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T10:44:27.527625+0000 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-25T10:53:31.142322+0000 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 26 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-04-25T10:54:17.247 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/2420046240,v1:172.21.15.19:6803/2420046240] [v2:172.21.15.19:6804/2420046240,v1:172.21.15.19:6805/2420046240] exists,up d1a36b37-3389-4f73-873b-7dd279f1701a 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6800/3663453616,v1:172.21.15.112:6801/3663453616] [v2:172.21.15.112:6802/3663453616,v1:172.21.15.112:6803/3663453616] exists,up d41a19a6-e1ab-4d69-87da-3ce650ec50c0 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6800/3608720428,v1:172.21.15.169:6801/3608720428] [v2:172.21.15.169:6802/3608720428,v1:172.21.15.169:6803/3608720428] exists,up d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 26 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/930248439,v1:172.21.15.19:6811/930248439] [v2:172.21.15.19:6812/930248439,v1:172.21.15.19:6813/930248439] exists,up efc21e6a-f4d3-4abd-aee6-b6a8533142e1 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 32 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6808/3068928119,v1:172.21.15.112:6809/3068928119] [v2:172.21.15.112:6810/3068928119,v1:172.21.15.112:6811/3068928119] exists,up a1ec5ae4-0515-4c9d-a701-7ea0e010534f 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 37 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6808/3010722907,v1:172.21.15.169:6809/3010722907] [v2:172.21.15.169:6810/3010722907,v1:172.21.15.169:6811/3010722907] exists,up 6c271d7f-d82b-4849-a765-f0d0de0ad3db 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 43 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/2613393193,v1:172.21.15.19:6819/2613393193] [v2:172.21.15.19:6820/2613393193,v1:172.21.15.19:6821/2613393193] exists,up 5a24aa3b-f969-467b-8fdf-f0550d37a071 2024-04-25T10:54:17.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 48 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6816/2341909802,v1:172.21.15.112:6817/2341909802] [v2:172.21.15.112:6818/2341909802,v1:172.21.15.112:6819/2341909802] exists,up 1325203b-15e0-415c-a705-0e6b38ef714a 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 54 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6816/4055159566,v1:172.21.15.169:6817/4055159566] [v2:172.21.15.169:6818/4055159566,v1:172.21.15.169:6819/4055159566] exists,up 45140686-3fd7-44dd-a246-d67730c2b8c3 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 60 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/920660159,v1:172.21.15.19:6827/920660159] [v2:172.21.15.19:6828/920660159,v1:172.21.15.19:6829/920660159] exists,up 1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 66 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6824/2703835627,v1:172.21.15.112:6825/2703835627] [v2:172.21.15.112:6826/2703835627,v1:172.21.15.112:6827/2703835627] exists,up 6301d36c-b3e4-4840-bde6-833e2da34788 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 72 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6824/2887715484,v1:172.21.15.169:6825/2887715484] [v2:172.21.15.169:6826/2887715484,v1:172.21.15.169:6827/2887715484] exists,up 9479e8e3-d6cf-4826-8411-9c6801ec4fd3 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.8 [2,11] 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1c [8,11] 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.0 [5,9] 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.c [3,9] 2024-04-25T10:54:17.249 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.11 [4,7] 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1b [4,8] 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1d [4,6] 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.33 [5,9] 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/1932674296 expires 2024-04-26T10:45:29.902440+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/1932674296 expires 2024-04-26T10:45:29.902440+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1427296 expires 2024-04-26T10:45:29.902440+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3362794439 expires 2024-04-26T10:45:29.902440+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/370155901 expires 2024-04-26T10:45:01.574319+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1990434922 expires 2024-04-26T10:45:01.574319+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/370155901 expires 2024-04-26T10:45:01.574319+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/111580854 expires 2024-04-26T10:45:01.574319+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/881789175 expires 2024-04-26T10:45:01.574319+0000 2024-04-25T10:54:17.250 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1153623604 expires 2024-04-26T10:45:29.902440+0000 2024-04-25T10:54:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63700afb80 msgr2=0x7f63700b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63700afb80 0x7f63700b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f638819dbe0 tx=0x7f637c009190 comp rx=0 tx=0).stop 2024-04-25T10:54:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388103120 msgr2=0x7f638819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388103120 0x7f638819cb30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f637800b4d0 tx=0x7f637800b7e0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 shutdown_connections 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f63700afb80 0x7f63700b2090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6388104bf0 0x7f63881a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6388103120 0x7f638819cb30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 --2- 172.21.15.19:0/1226753227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6388104250 0x7f638819d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 >> 172.21.15.19:0/1226753227 conn(0x7f63880fe760 msgr2=0x7f63880758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 shutdown_connections 2024-04-25T10:54:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:17.248+0000 7f638e86d700 1 -- 172.21.15.19:0/1226753227 wait complete. 2024-04-25T10:54:17.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:17 smithi019 ceph-mon[29209]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-25T10:54:17.596 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:17 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1226753227' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T10:54:17.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:17 smithi169 ceph-mon[33049]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-25T10:54:17.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:17 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1226753227' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T10:54:17.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:17 smithi112 ceph-mon[32799]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-25T10:54:17.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:17 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1226753227' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T10:54:17.892 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T10:54:17.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:54:18.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 -- 172.21.15.19:0/496001292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa901029c0 msgr2=0x7faa901036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 --2- 172.21.15.19:0/496001292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa901029c0 0x7faa901036c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa88009900 tx=0x7faa8802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:18.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 -- 172.21.15.19:0/496001292 shutdown_connections 2024-04-25T10:54:18.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 --2- 172.21.15.19:0/496001292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa90108900 0x7faa9010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 --2- 172.21.15.19:0/496001292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa90103c00 0x7faa90104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 --2- 172.21.15.19:0/496001292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa901029c0 0x7faa901036c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 -- 172.21.15.19:0/496001292 >> 172.21.15.19:0/496001292 conn(0x7faa900fdbd0 msgr2=0x7faa90100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:18.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 -- 172.21.15.19:0/496001292 shutdown_connections 2024-04-25T10:54:18.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.020+0000 7faa95661640 1 -- 172.21.15.19:0/496001292 wait complete. 2024-04-25T10:54:18.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 Processor -- start 2024-04-25T10:54:18.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 -- start start 2024-04-25T10:54:18.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa901029c0 0x7faa90199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa90103c00 0x7faa901996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa90108900 0x7faa901a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa9010e4c0 con 0x7faa90103c00 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa9010e340 con 0x7faa901029c0 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.021+0000 7faa95661640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa9010e640 con 0x7faa90108900 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa90103c00 0x7faa901996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa90103c00 0x7faa901996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43956/0 (socket says 172.21.15.19:43956) 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 -- 172.21.15.19:0/3209971673 learned_addr learned my addr 172.21.15.19:0/3209971673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8f7fe640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa90108900 0x7faa901a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8effd640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa901029c0 0x7faa90199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 -- 172.21.15.19:0/3209971673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa90108900 msgr2=0x7faa901a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa90108900 0x7faa901a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 -- 172.21.15.19:0/3209971673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa901029c0 msgr2=0x7faa90199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa901029c0 0x7faa90199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 -- 172.21.15.19:0/3209971673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa901a0c70 con 0x7faa90103c00 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8f7fe640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa90108900 0x7faa901a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa8e7fc640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa90103c00 0x7faa901996c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faa7800d770 tx=0x7faa7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa87fff640 1 -- 172.21.15.19:0/3209971673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78017070 con 0x7faa90103c00 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa87fff640 1 -- 172.21.15.19:0/3209971673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa780054d0 con 0x7faa90103c00 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa901a0f60 con 0x7faa90103c00 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa87fff640 1 -- 172.21.15.19:0/3209971673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78004c30 con 0x7faa90103c00 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.022+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a13c0 con 0x7faa90103c00 2024-04-25T10:54:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.023+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa90107710 con 0x7faa90103c00 2024-04-25T10:54:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.025+0000 7faa87fff640 1 -- 172.21.15.19:0/3209971673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7faa78004dd0 con 0x7faa90103c00 2024-04-25T10:54:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.025+0000 7faa87fff640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faa7c0afaf0 0x7faa7c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.025+0000 7faa87fff640 1 -- 172.21.15.19:0/3209971673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7faa780deb90 con 0x7faa90103c00 2024-04-25T10:54:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.025+0000 7faa8effd640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faa7c0afaf0 0x7faa7c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.026+0000 7faa8effd640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faa7c0afaf0 0x7faa7c0b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faa8802a930 tx=0x7faa88005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.029+0000 7faa87fff640 1 -- 172.21.15.19:0/3209971673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa780a7780 con 0x7faa90103c00 2024-04-25T10:54:18.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.216+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa900637d0 con 0x7faa90103c00 2024-04-25T10:54:18.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.218+0000 7faa87fff640 1 -- 172.21.15.19:0/3209971673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7faa780a7080 con 0x7faa90103c00 2024-04-25T10:54:18.219 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:18.219 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faa7c0afaf0 msgr2=0x7faa7c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faa7c0afaf0 0x7faa7c0b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faa8802a930 tx=0x7faa88005fd0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa90103c00 msgr2=0x7faa901996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa90103c00 0x7faa901996c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faa7800d770 tx=0x7faa7800dc30 comp rx=0 tx=0).stop 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 shutdown_connections 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa90108900 0x7faa901a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7faa7c0afaf0 0x7faa7c0b2000 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa90103c00 0x7faa901996c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 --2- 172.21.15.19:0/3209971673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa901029c0 0x7faa90199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 >> 172.21.15.19:0/3209971673 conn(0x7faa900fdbd0 msgr2=0x7faa901048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:18.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 shutdown_connections 2024-04-25T10:54:18.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.220+0000 7faa95661640 1 -- 172.21.15.19:0/3209971673 wait complete. 2024-04-25T10:54:18.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:54:18.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:54:18.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 -- 172.21.15.19:0/288987561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc401029e0 msgr2=0x7fbc401036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 --2- 172.21.15.19:0/288987561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc401029e0 0x7fbc401036e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009900 tx=0x7fbc2802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:18.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 -- 172.21.15.19:0/288987561 shutdown_connections 2024-04-25T10:54:18.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 --2- 172.21.15.19:0/288987561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc40108920 0x7fbc4010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 --2- 172.21.15.19:0/288987561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc40103c20 0x7fbc40104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 --2- 172.21.15.19:0/288987561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc401029e0 0x7fbc401036e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 -- 172.21.15.19:0/288987561 >> 172.21.15.19:0/288987561 conn(0x7fbc400fdbd0 msgr2=0x7fbc40100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 -- 172.21.15.19:0/288987561 shutdown_connections 2024-04-25T10:54:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.351+0000 7fbc44b14640 1 -- 172.21.15.19:0/288987561 wait complete. 2024-04-25T10:54:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 Processor -- start 2024-04-25T10:54:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 -- start start 2024-04-25T10:54:18.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc401029e0 0x7fbc4019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc40103c20 0x7fbc4019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc40108920 0x7fbc401a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4010e510 con 0x7fbc40108920 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4010e390 con 0x7fbc40103c20 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc44b14640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4010e690 con 0x7fbc401029e0 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc40108920 0x7fbc401a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc40108920 0x7fbc401a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43976/0 (socket says 172.21.15.19:43976) 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 -- 172.21.15.19:0/3124408888 learned_addr learned my addr 172.21.15.19:0/3124408888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3dd9b640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc40103c20 0x7fbc4019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3e59c640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc401029e0 0x7fbc4019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 -- 172.21.15.19:0/3124408888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc401029e0 msgr2=0x7fbc4019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc401029e0 0x7fbc4019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 -- 172.21.15.19:0/3124408888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc40103c20 msgr2=0x7fbc4019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc40103c20 0x7fbc4019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.352+0000 7fbc3ed9d640 1 -- 172.21.15.19:0/3124408888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc401a1ff0 con 0x7fbc40108920 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.353+0000 7fbc3e59c640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc401029e0 0x7fbc4019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.353+0000 7fbc3ed9d640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc40108920 0x7fbc401a1a30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cad0 tx=0x7fbc3800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.353+0000 7fbc377fe640 1 -- 172.21.15.19:0/3124408888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800ad60 con 0x7fbc40108920 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.353+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc401a22e0 con 0x7fbc40108920 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.353+0000 7fbc377fe640 1 -- 172.21.15.19:0/3124408888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38005020 con 0x7fbc40108920 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.353+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc401a2830 con 0x7fbc40108920 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.353+0000 7fbc377fe640 1 -- 172.21.15.19:0/3124408888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800f6b0 con 0x7fbc40108920 2024-04-25T10:54:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.354+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4006c2a0 con 0x7fbc40108920 2024-04-25T10:54:18.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.355+0000 7fbc377fe640 1 -- 172.21.15.19:0/3124408888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbc380051d0 con 0x7fbc40108920 2024-04-25T10:54:18.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.356+0000 7fbc377fe640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbc2c0afaf0 0x7fbc2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.356+0000 7fbc3e59c640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbc2c0afaf0 0x7fbc2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.356+0000 7fbc377fe640 1 -- 172.21.15.19:0/3124408888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fbc380def90 con 0x7fbc40108920 2024-04-25T10:54:18.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.357+0000 7fbc3e59c640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbc2c0afaf0 0x7fbc2c0b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc2802aa00 tx=0x7fbc28031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.359+0000 7fbc377fe640 1 -- 172.21.15.19:0/3124408888 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc38014030 con 0x7fbc40108920 2024-04-25T10:54:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.545+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc401a2f50 con 0x7fbc40108920 2024-04-25T10:54:18.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.545+0000 7fbc377fe640 1 -- 172.21.15.19:0/3124408888 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fbc380a7c20 con 0x7fbc40108920 2024-04-25T10:54:18.546 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbc2c0afaf0 msgr2=0x7fbc2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbc2c0afaf0 0x7fbc2c0b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc2802aa00 tx=0x7fbc28031400 comp rx=0 tx=0).stop 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc40108920 msgr2=0x7fbc401a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc40108920 0x7fbc401a1a30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cad0 tx=0x7fbc3800ced0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 shutdown_connections 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fbc2c0afaf0 0x7fbc2c0b2000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc40108920 0x7fbc401a1a30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc40103c20 0x7fbc4019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 --2- 172.21.15.19:0/3124408888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc401029e0 0x7fbc4019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 >> 172.21.15.19:0/3124408888 conn(0x7fbc400fdbd0 msgr2=0x7fbc40109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:18.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 shutdown_connections 2024-04-25T10:54:18.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.547+0000 7fbc44b14640 1 -- 172.21.15.19:0/3124408888 wait complete. 2024-04-25T10:54:18.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:18 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3209971673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:18.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 -- 172.21.15.19:0/147005534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c109a40 msgr2=0x7f149c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 --2- 172.21.15.19:0/147005534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c109a40 0x7f149c10be60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1494009ca0 tx=0x7f149402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 -- 172.21.15.19:0/147005534 shutdown_connections 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 --2- 172.21.15.19:0/147005534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c109a40 0x7f149c10be60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 --2- 172.21.15.19:0/147005534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c104d40 0x7f149c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 --2- 172.21.15.19:0/147005534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c103b00 0x7f149c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 -- 172.21.15.19:0/147005534 >> 172.21.15.19:0/147005534 conn(0x7f149c0fecf0 msgr2=0x7f149c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 -- 172.21.15.19:0/147005534 shutdown_connections 2024-04-25T10:54:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.674+0000 7f14a0b72640 1 -- 172.21.15.19:0/147005534 wait complete. 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 Processor -- start 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 -- start start 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c103b00 0x7f149c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c104d40 0x7f149c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c109a40 0x7f149c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f149c10f630 con 0x7f149c104d40 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f149c10f4b0 con 0x7f149c103b00 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f14a0b72640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f149c10f7b0 con 0x7f149c109a40 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f1499d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c104d40 0x7f149c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f1499d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c104d40 0x7f149c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44000/0 (socket says 172.21.15.19:44000) 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f1499d9b640 1 -- 172.21.15.19:0/2848571970 learned_addr learned my addr 172.21.15.19:0/2848571970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.675+0000 7f149ad9d640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c109a40 0x7f149c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f149a59c640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c103b00 0x7f149c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f1499d9b640 1 -- 172.21.15.19:0/2848571970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c109a40 msgr2=0x7f149c1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f1499d9b640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c109a40 0x7f149c1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f1499d9b640 1 -- 172.21.15.19:0/2848571970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c103b00 msgr2=0x7f149c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f1499d9b640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c103b00 0x7f149c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f1499d9b640 1 -- 172.21.15.19:0/2848571970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c1a1d10 con 0x7f149c104d40 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f1499d9b640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c104d40 0x7f149c19a710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f149000d950 tx=0x7f149000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f148b7fe640 1 -- 172.21.15.19:0/2848571970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490017070 con 0x7f149c104d40 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149c1a1fa0 con 0x7f149c104d40 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f148b7fe640 1 -- 172.21.15.19:0/2848571970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1490009c30 con 0x7f149c104d40 2024-04-25T10:54:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f148b7fe640 1 -- 172.21.15.19:0/2848571970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149000f620 con 0x7f149c104d40 2024-04-25T10:54:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.676+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149c1a2490 con 0x7f149c104d40 2024-04-25T10:54:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.677+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149c108850 con 0x7f149c104d40 2024-04-25T10:54:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.678+0000 7f148b7fe640 1 -- 172.21.15.19:0/2848571970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1490010460 con 0x7f149c104d40 2024-04-25T10:54:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.679+0000 7f148b7fe640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f14840afaf0 0x7f14840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.679+0000 7f149a59c640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f14840afaf0 0x7f14840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.679+0000 7f148b7fe640 1 -- 172.21.15.19:0/2848571970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f14900de930 con 0x7f149c104d40 2024-04-25T10:54:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.680+0000 7f149a59c640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f14840afaf0 0x7f14840b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f149c104700 tx=0x7f148c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:18.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.684+0000 7f148b7fe640 1 -- 172.21.15.19:0/2848571970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14900a7520 con 0x7f149c104d40 2024-04-25T10:54:18.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:18 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3209971673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:18.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:18 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3209971673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:18.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.869+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f149c02db50 con 0x7f149c104d40 2024-04-25T10:54:18.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.870+0000 7f148b7fe640 1 -- 172.21.15.19:0/2848571970 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f1490009de0 con 0x7f149c104d40 2024-04-25T10:54:18.870 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:54:18.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.871+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f14840afaf0 msgr2=0x7f14840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.871+0000 7f14a0b72640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f14840afaf0 0x7f14840b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f149c104700 tx=0x7f148c005f90 comp rx=0 tx=0).stop 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.871+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c104d40 msgr2=0x7f149c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.871+0000 7f14a0b72640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c104d40 0x7f149c19a710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f149000d950 tx=0x7f149000de10 comp rx=0 tx=0).stop 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 shutdown_connections 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c109a40 0x7f149c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f14840afaf0 0x7f14840b2000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c104d40 0x7f149c19a710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 --2- 172.21.15.19:0/2848571970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c103b00 0x7f149c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 >> 172.21.15.19:0/2848571970 conn(0x7f149c0fecf0 msgr2=0x7f149c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 shutdown_connections 2024-04-25T10:54:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.872+0000 7f14a0b72640 1 -- 172.21.15.19:0/2848571970 wait complete. 2024-04-25T10:54:18.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.997+0000 7f847d5c5640 1 -- 172.21.15.19:0/578144090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84781045d0 msgr2=0x7f84781069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.997+0000 7f847d5c5640 1 --2- 172.21.15.19:0/578144090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84781045d0 0x7f84781069f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8468009900 tx=0x7f846802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.997+0000 7f847d5c5640 1 -- 172.21.15.19:0/578144090 shutdown_connections 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.997+0000 7f847d5c5640 1 --2- 172.21.15.19:0/578144090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478109890 0x7f847810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.997+0000 7f847d5c5640 1 --2- 172.21.15.19:0/578144090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478106f30 0x7f8478109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.997+0000 7f847d5c5640 1 --2- 172.21.15.19:0/578144090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84781045d0 0x7f84781069f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.997+0000 7f847d5c5640 1 -- 172.21.15.19:0/578144090 >> 172.21.15.19:0/578144090 conn(0x7f84780fdbd0 msgr2=0x7f8478100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.998+0000 7f847d5c5640 1 -- 172.21.15.19:0/578144090 shutdown_connections 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.998+0000 7f847d5c5640 1 -- 172.21.15.19:0/578144090 wait complete. 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.998+0000 7f847d5c5640 1 Processor -- start 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.998+0000 7f847d5c5640 1 -- start start 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.998+0000 7f847d5c5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84781045d0 0x7f847819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f847d5c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478106f30 0x7f847819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f847d5c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8478109890 0x7f84781a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f847d5c5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f847810f5e0 con 0x7f8478109890 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f847d5c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f847810f460 con 0x7f8478106f30 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f847d5c5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f847810f760 con 0x7f84781045d0 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8478109890 0x7f84781a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8478109890 0x7f84781a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44022/0 (socket says 172.21.15.19:44022) 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 -- 172.21.15.19:0/3386012852 learned_addr learned my addr 172.21.15.19:0/3386012852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84767fc640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478106f30 0x7f847819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f8476ffd640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84781045d0 0x7f847819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 -- 172.21.15.19:0/3386012852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84781045d0 msgr2=0x7f847819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84781045d0 0x7f847819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 -- 172.21.15.19:0/3386012852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478106f30 msgr2=0x7f847819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478106f30 0x7f847819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 -- 172.21.15.19:0/3386012852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84781a42d0 con 0x7f8478109890 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f8476ffd640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84781045d0 0x7f847819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84767fc640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478106f30 0x7f847819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f84777fe640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8478109890 0x7f84781a3c60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f847000cad0 tx=0x7f847000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f845ffff640 1 -- 172.21.15.19:0/3386012852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847000ae20 con 0x7f8478109890 2024-04-25T10:54:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84781a45c0 con 0x7f8478109890 2024-04-25T10:54:19.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f845ffff640 1 -- 172.21.15.19:0/3386012852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8470003e40 con 0x7f8478109890 2024-04-25T10:54:19.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:18.999+0000 7f845ffff640 1 -- 172.21.15.19:0/3386012852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847000f6b0 con 0x7f8478109890 2024-04-25T10:54:19.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.000+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84781a4b60 con 0x7f8478109890 2024-04-25T10:54:19.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.001+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847806c2a0 con 0x7f8478109890 2024-04-25T10:54:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.002+0000 7f845ffff640 1 -- 172.21.15.19:0/3386012852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f84700042c0 con 0x7f8478109890 2024-04-25T10:54:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.003+0000 7f845ffff640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84600afaf0 0x7f84600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.003+0000 7f8476ffd640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84600afaf0 0x7f84600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.003+0000 7f845ffff640 1 -- 172.21.15.19:0/3386012852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f84700df070 con 0x7f8478109890 2024-04-25T10:54:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.003+0000 7f8476ffd640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84600afaf0 0x7f84600b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8468005f70 tx=0x7f8468000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.007+0000 7f845ffff640 1 -- 172.21.15.19:0/3386012852 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8470014070 con 0x7f8478109890 2024-04-25T10:54:19.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.195+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f84781a51b0 con 0x7f8478109890 2024-04-25T10:54:19.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.195+0000 7f845ffff640 1 -- 172.21.15.19:0/3386012852 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f84700a7c00 con 0x7f8478109890 2024-04-25T10:54:19.199 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84600afaf0 msgr2=0x7f84600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84600afaf0 0x7f84600b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8468005f70 tx=0x7f8468000bc0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8478109890 msgr2=0x7f84781a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8478109890 0x7f84781a3c60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f847000cad0 tx=0x7f847000cf90 comp rx=0 tx=0).stop 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 shutdown_connections 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f84600afaf0 0x7f84600b2000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8478109890 0x7f84781a3c60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478106f30 0x7f847819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 --2- 172.21.15.19:0/3386012852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84781045d0 0x7f847819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 >> 172.21.15.19:0/3386012852 conn(0x7f84780fdbd0 msgr2=0x7f8478102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 shutdown_connections 2024-04-25T10:54:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.197+0000 7f847d5c5640 1 -- 172.21.15.19:0/3386012852 wait complete. 2024-04-25T10:54:19.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:54:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 -- 172.21.15.19:0/3650146084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec103c20 msgr2=0x7fadec104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 --2- 172.21.15.19:0/3650146084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec103c20 0x7fadec104030 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fade0009900 tx=0x7fade002a3f0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 -- 172.21.15.19:0/3650146084 shutdown_connections 2024-04-25T10:54:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 --2- 172.21.15.19:0/3650146084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadec108920 0x7fadec10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 --2- 172.21.15.19:0/3650146084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec103c20 0x7fadec104030 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 --2- 172.21.15.19:0/3650146084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadec1029e0 0x7fadec1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 -- 172.21.15.19:0/3650146084 >> 172.21.15.19:0/3650146084 conn(0x7fadec0fdbd0 msgr2=0x7fadec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 -- 172.21.15.19:0/3650146084 shutdown_connections 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.321+0000 7fadf29d5640 1 -- 172.21.15.19:0/3650146084 wait complete. 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 Processor -- start 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 -- start start 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadec1029e0 0x7fadec075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadec103c20 0x7fadec19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec108920 0x7fadec1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10e4e0 con 0x7fadec108920 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10e360 con 0x7fadec1029e0 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.322+0000 7fadf29d5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10e660 con 0x7fadec103c20 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec108920 0x7fadec1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec108920 0x7fadec1a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44050/0 (socket says 172.21.15.19:44050) 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 -- 172.21.15.19:0/1225185729 learned_addr learned my addr 172.21.15.19:0/1225185729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadebfff640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadec1029e0 0x7fadec075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadeb7fe640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadec103c20 0x7fadec19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 -- 172.21.15.19:0/1225185729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadec103c20 msgr2=0x7fadec19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadec103c20 0x7fadec19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 -- 172.21.15.19:0/1225185729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadec1029e0 msgr2=0x7fadec075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadec1029e0 0x7fadec075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 -- 172.21.15.19:0/1225185729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadec1a1c70 con 0x7fadec108920 2024-04-25T10:54:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadebfff640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadec1029e0 0x7fadec075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf0f72640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec108920 0x7fadec1a14e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fade400cce0 tx=0x7fade400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fade97fa640 1 -- 172.21.15.19:0/1225185729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade400f450 con 0x7fadec108920 2024-04-25T10:54:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fade97fa640 1 -- 172.21.15.19:0/1225185729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade400faa0 con 0x7fadec108920 2024-04-25T10:54:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadec1a1f60 con 0x7fadec108920 2024-04-25T10:54:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fade97fa640 1 -- 172.21.15.19:0/1225185729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4018b70 con 0x7fadec108920 2024-04-25T10:54:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.323+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadec1a2370 con 0x7fadec108920 2024-04-25T10:54:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.324+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadec06a1c0 con 0x7fadec108920 2024-04-25T10:54:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.327+0000 7fade97fa640 1 -- 172.21.15.19:0/1225185729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fade4018d10 con 0x7fadec108920 2024-04-25T10:54:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.327+0000 7fade97fa640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fadd40afaf0 0x7fadd40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.327+0000 7fadebfff640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fadd40afaf0 0x7fadd40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.327+0000 7fade97fa640 1 -- 172.21.15.19:0/1225185729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fade40df1e0 con 0x7fadec108920 2024-04-25T10:54:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.328+0000 7fadebfff640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fadd40afaf0 0x7fadd40b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fadec1035e0 tx=0x7faddc005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.331+0000 7fade97fa640 1 -- 172.21.15.19:0/1225185729 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadec06a1c0 con 0x7fadec108920 2024-04-25T10:54:19.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.555+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadec02dc10 con 0x7fadec108920 2024-04-25T10:54:19.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.556+0000 7fade97fa640 1 -- 172.21.15.19:0/1225185729 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fade4014030 con 0x7fadec108920 2024-04-25T10:54:19.557 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:19.557 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:54:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fadd40afaf0 msgr2=0x7fadd40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fadd40afaf0 0x7fadd40b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fadec1035e0 tx=0x7faddc005db0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec108920 msgr2=0x7fadec1a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec108920 0x7fadec1a14e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fade400cce0 tx=0x7fade400a620 comp rx=0 tx=0).stop 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 shutdown_connections 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fadd40afaf0 0x7fadd40b2000 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadec108920 0x7fadec1a14e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadec103c20 0x7fadec19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 --2- 172.21.15.19:0/1225185729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadec1029e0 0x7fadec075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 >> 172.21.15.19:0/1225185729 conn(0x7fadec0fdbd0 msgr2=0x7fadec104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 shutdown_connections 2024-04-25T10:54:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.558+0000 7fadf29d5640 1 -- 172.21.15.19:0/1225185729 wait complete. 2024-04-25T10:54:19.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:19 smithi019 ceph-mon[29209]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:19.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:19 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3124408888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:19 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2848571970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:19 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3386012852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.570 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:54:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 -- 172.21.15.19:0/3513275261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 msgr2=0x7f86741036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 --2- 172.21.15.19:0/3513275261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f86741036c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8664009900 tx=0x7f866402a3f0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 -- 172.21.15.19:0/3513275261 shutdown_connections 2024-04-25T10:54:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 --2- 172.21.15.19:0/3513275261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8674108900 0x7f867410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 --2- 172.21.15.19:0/3513275261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8674103c00 0x7f8674104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 --2- 172.21.15.19:0/3513275261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f86741036c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 -- 172.21.15.19:0/3513275261 >> 172.21.15.19:0/3513275261 conn(0x7f86740fdbd0 msgr2=0x7f8674100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 -- 172.21.15.19:0/3513275261 shutdown_connections 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.682+0000 7f8678cbd640 1 -- 172.21.15.19:0/3513275261 wait complete. 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 Processor -- start 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 -- start start 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f867419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8674103c00 0x7f867419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8674108900 0x7f86741a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f867410f5f0 con 0x7f86741029c0 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f867410f470 con 0x7f8674103c00 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8678cbd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f867410f770 con 0x7f8674108900 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f867419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f867419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44072/0 (socket says 172.21.15.19:44072) 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 -- 172.21.15.19:0/603289335 learned_addr learned my addr 172.21.15.19:0/603289335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8671d9b640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8674103c00 0x7f867419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8672d9d640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8674108900 0x7f86741a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 -- 172.21.15.19:0/603289335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8674108900 msgr2=0x7f86741a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8674108900 0x7f86741a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 -- 172.21.15.19:0/603289335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8674103c00 msgr2=0x7f867419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8674103c00 0x7f867419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 -- 172.21.15.19:0/603289335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86741a1c30 con 0x7f86741029c0 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8671d9b640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8674103c00 0x7f867419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f8672d9d640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8674108900 0x7f86741a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.683+0000 7f867259c640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f867419a210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8664032fd0 tx=0x7f86640042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.684+0000 7f86637fe640 1 -- 172.21.15.19:0/603289335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866403f070 con 0x7f86741029c0 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.684+0000 7f86637fe640 1 -- 172.21.15.19:0/603289335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86640044d0 con 0x7f86741029c0 2024-04-25T10:54:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.684+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86741a1ec0 con 0x7f86741029c0 2024-04-25T10:54:19.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.684+0000 7f86637fe640 1 -- 172.21.15.19:0/603289335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664035070 con 0x7f86741029c0 2024-04-25T10:54:19.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.684+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86741a2320 con 0x7f86741029c0 2024-04-25T10:54:19.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.685+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8674107710 con 0x7f86741029c0 2024-04-25T10:54:19.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.686+0000 7f86637fe640 1 -- 172.21.15.19:0/603289335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f866403b5f0 con 0x7f86741029c0 2024-04-25T10:54:19.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.687+0000 7f86637fe640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f865c0afb70 0x7f865c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:19.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.687+0000 7f8671d9b640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f865c0afb70 0x7f865c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:19.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.687+0000 7f86637fe640 1 -- 172.21.15.19:0/603289335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f86640ff8a0 con 0x7f86741029c0 2024-04-25T10:54:19.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.687+0000 7f8671d9b640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f865c0afb70 0x7f865c0b2080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8668005fd0 tx=0x7f8668005f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:19.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.691+0000 7f86637fe640 1 -- 172.21.15.19:0/603289335 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86640c8490 con 0x7f86741029c0 2024-04-25T10:54:19.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:19 smithi169 ceph-mon[33049]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:19.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:19 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3124408888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:19 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2848571970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:19 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3386012852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:19 smithi112 ceph-mon[32799]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:19.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:19 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3124408888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:19 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2848571970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:19 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3386012852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:19.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.879+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f867402dd20 con 0x7f86741029c0 2024-04-25T10:54:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.880+0000 7f86637fe640 1 -- 172.21.15.19:0/603289335 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7f86640c7d90 con 0x7f86741029c0 2024-04-25T10:54:19.881 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:19.882 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:54:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f865c0afb70 msgr2=0x7f865c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f865c0afb70 0x7f865c0b2080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8668005fd0 tx=0x7f8668005f40 comp rx=0 tx=0).stop 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 msgr2=0x7f867419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f867419a210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8664032fd0 tx=0x7f86640042e0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 shutdown_connections 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8674108900 0x7f86741a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8674103c00 0x7f867419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f865c0afb70 0x7f865c0b2080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 --2- 172.21.15.19:0/603289335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86741029c0 0x7f867419a210 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 >> 172.21.15.19:0/603289335 conn(0x7f86740fdbd0 msgr2=0x7f86741048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 shutdown_connections 2024-04-25T10:54:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:19.883+0000 7f8678cbd640 1 -- 172.21.15.19:0/603289335 wait complete. 2024-04-25T10:54:19.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:54:19.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 -- 172.21.15.19:0/476132221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c103c30 msgr2=0x7f346c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 --2- 172.21.15.19:0/476132221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c103c30 0x7f346c104040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3460009900 tx=0x7f346002a3f0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 -- 172.21.15.19:0/476132221 shutdown_connections 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 --2- 172.21.15.19:0/476132221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c108930 0x7f346c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 --2- 172.21.15.19:0/476132221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c103c30 0x7f346c104040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 --2- 172.21.15.19:0/476132221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c1029f0 0x7f346c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 -- 172.21.15.19:0/476132221 >> 172.21.15.19:0/476132221 conn(0x7f346c0fdbc0 msgr2=0x7f346c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 -- 172.21.15.19:0/476132221 shutdown_connections 2024-04-25T10:54:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 -- 172.21.15.19:0/476132221 wait complete. 2024-04-25T10:54:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.010+0000 7f3471251640 1 Processor -- start 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f3471251640 1 -- start start 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f3471251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c1029f0 0x7f346c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f3471251640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c103c30 0x7f346c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f3471251640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c108930 0x7f346c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f3471251640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f346c10f620 con 0x7f346c1029f0 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f3471251640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f346c10f4a0 con 0x7f346c108930 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f3471251640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f346c10f7a0 con 0x7f346c103c30 2024-04-25T10:54:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c1029f0 0x7f346c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c1029f0 0x7f346c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44088/0 (socket says 172.21.15.19:44088) 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 -- 172.21.15.19:0/1455279350 learned_addr learned my addr 172.21.15.19:0/1455279350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346a59c640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c103c30 0x7f346c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346b59e640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c108930 0x7f346c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 -- 172.21.15.19:0/1455279350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c103c30 msgr2=0x7f346c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c103c30 0x7f346c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 -- 172.21.15.19:0/1455279350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c108930 msgr2=0x7f346c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c108930 0x7f346c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.011+0000 7f346ad9d640 1 -- 172.21.15.19:0/1455279350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c1a1cc0 con 0x7f346c1029f0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.012+0000 7f346ad9d640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c1029f0 0x7f346c19a230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f345c00c970 tx=0x7f345c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.012+0000 7f3453fff640 1 -- 172.21.15.19:0/1455279350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c007c70 con 0x7f346c1029f0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.012+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c1a1fb0 con 0x7f346c1029f0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.012+0000 7f3453fff640 1 -- 172.21.15.19:0/1455279350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c007e10 con 0x7f346c1029f0 2024-04-25T10:54:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.012+0000 7f3453fff640 1 -- 172.21.15.19:0/1455279350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0187b0 con 0x7f346c1029f0 2024-04-25T10:54:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.012+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a2410 con 0x7f346c1029f0 2024-04-25T10:54:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.013+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346c06a190 con 0x7f346c1029f0 2024-04-25T10:54:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.014+0000 7f3453fff640 1 -- 172.21.15.19:0/1455279350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f345c00fab0 con 0x7f346c1029f0 2024-04-25T10:54:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.015+0000 7f3453fff640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f34540afaf0 0x7f34540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.015+0000 7f346a59c640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f34540afaf0 0x7f34540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.015+0000 7f3453fff640 1 -- 172.21.15.19:0/1455279350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f345c0def30 con 0x7f346c1029f0 2024-04-25T10:54:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.016+0000 7f346a59c640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f34540afaf0 0x7f34540b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3460000c00 tx=0x7f3460031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.018+0000 7f3453fff640 1 -- 172.21.15.19:0/1455279350 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f345c053ff0 con 0x7f346c1029f0 2024-04-25T10:54:20.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.206+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f346c02dbf0 con 0x7f346c1029f0 2024-04-25T10:54:20.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.206+0000 7f3453fff640 1 -- 172.21.15.19:0/1455279350 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f346c06a190 con 0x7f346c1029f0 2024-04-25T10:54:20.207 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f34540afaf0 msgr2=0x7f34540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f34540afaf0 0x7f34540b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3460000c00 tx=0x7f3460031400 comp rx=0 tx=0).stop 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c1029f0 msgr2=0x7f346c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c1029f0 0x7f346c19a230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f345c00c970 tx=0x7f345c00ce30 comp rx=0 tx=0).stop 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 shutdown_connections 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c108930 0x7f346c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c103c30 0x7f346c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f34540afaf0 0x7f34540b2000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 --2- 172.21.15.19:0/1455279350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c1029f0 0x7f346c19a230 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 >> 172.21.15.19:0/1455279350 conn(0x7f346c0fdbc0 msgr2=0x7f346c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 shutdown_connections 2024-04-25T10:54:20.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.208+0000 7f3471251640 1 -- 172.21.15.19:0/1455279350 wait complete. 2024-04-25T10:54:20.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:54:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 -- 172.21.15.19:0/3800158153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 msgr2=0x7fcb8410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/3800158153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb8410bcc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009ca0 tx=0x7fcb7c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 -- 172.21.15.19:0/3800158153 shutdown_connections 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/3800158153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb8410bcc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/3800158153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb84106f40 0x7fcb84109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/3800158153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb841045e0 0x7fcb84106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 -- 172.21.15.19:0/3800158153 >> 172.21.15.19:0/3800158153 conn(0x7fcb840fdbc0 msgr2=0x7fcb84100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 -- 172.21.15.19:0/3800158153 shutdown_connections 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.337+0000 7fcb8ae96640 1 -- 172.21.15.19:0/3800158153 wait complete. 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 Processor -- start 2024-04-25T10:54:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 -- start start 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb841045e0 0x7fcb84103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb84106f40 0x7fcb841020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb84102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8410f6f0 con 0x7fcb841098a0 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8410f570 con 0x7fcb84106f40 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb8ae96640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8410f870 con 0x7fcb841045e0 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb89433640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb84102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb89433640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb84102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44108/0 (socket says 172.21.15.19:44108) 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb89433640 1 -- 172.21.15.19:0/2219626543 learned_addr learned my addr 172.21.15.19:0/2219626543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb83fff640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb84106f40 0x7fcb841020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.338+0000 7fcb88c32640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb841045e0 0x7fcb84103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.339+0000 7fcb89433640 1 -- 172.21.15.19:0/2219626543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb841045e0 msgr2=0x7fcb84103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.339+0000 7fcb89433640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb841045e0 0x7fcb84103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.339+0000 7fcb89433640 1 -- 172.21.15.19:0/2219626543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb84106f40 msgr2=0x7fcb841020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.339+0000 7fcb89433640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb84106f40 0x7fcb841020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.339+0000 7fcb89433640 1 -- 172.21.15.19:0/2219626543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb84102d40 con 0x7fcb841098a0 2024-04-25T10:54:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.339+0000 7fcb89433640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb84102610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c02f500 tx=0x7fcb7c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.341+0000 7fcb81ffb640 1 -- 172.21.15.19:0/2219626543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0317c0 con 0x7fcb841098a0 2024-04-25T10:54:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.341+0000 7fcb81ffb640 1 -- 172.21.15.19:0/2219626543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7c0044f0 con 0x7fcb841098a0 2024-04-25T10:54:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.341+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb841a4410 con 0x7fcb841098a0 2024-04-25T10:54:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.341+0000 7fcb81ffb640 1 -- 172.21.15.19:0/2219626543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c02fdc0 con 0x7fcb841098a0 2024-04-25T10:54:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.341+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841a48f0 con 0x7fcb841098a0 2024-04-25T10:54:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.342+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8406a160 con 0x7fcb841098a0 2024-04-25T10:54:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.344+0000 7fcb81ffb640 1 -- 172.21.15.19:0/2219626543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcb7c004040 con 0x7fcb841098a0 2024-04-25T10:54:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.344+0000 7fcb81ffb640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb740afaf0 0x7fcb740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.345+0000 7fcb88c32640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb740afaf0 0x7fcb740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.345+0000 7fcb81ffb640 1 -- 172.21.15.19:0/2219626543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fcb7c100400 con 0x7fcb841098a0 2024-04-25T10:54:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.345+0000 7fcb88c32640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb740afaf0 0x7fcb740b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcb840757f0 tx=0x7fcb78009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.349+0000 7fcb81ffb640 1 -- 172.21.15.19:0/2219626543 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb7c0c8f90 con 0x7fcb841098a0 2024-04-25T10:54:20.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.534+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcb840637a0 con 0x7fcb841098a0 2024-04-25T10:54:20.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.535+0000 7fcb81ffb640 1 -- 172.21.15.19:0/2219626543 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fcb7c0c8890 con 0x7fcb841098a0 2024-04-25T10:54:20.536 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:54:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.537+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb740afaf0 msgr2=0x7fcb740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.537+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb740afaf0 0x7fcb740b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcb840757f0 tx=0x7fcb78009110 comp rx=0 tx=0).stop 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 msgr2=0x7fcb84102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb84102610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c02f500 tx=0x7fcb7c003ba0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 shutdown_connections 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb740afaf0 0x7fcb740b2000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb841098a0 0x7fcb84102610 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb84106f40 0x7fcb841020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 --2- 172.21.15.19:0/2219626543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb841045e0 0x7fcb84103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 >> 172.21.15.19:0/2219626543 conn(0x7fcb840fdbc0 msgr2=0x7fcb84105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 shutdown_connections 2024-04-25T10:54:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.538+0000 7fcb8ae96640 1 -- 172.21.15.19:0/2219626543 wait complete. 2024-04-25T10:54:20.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:20 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1225185729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:20.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:20 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/603289335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:20.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:20 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1455279350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:20.550 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:54:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 -- 172.21.15.19:0/1783302651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 msgr2=0x7f22f8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 --2- 172.21.15.19:0/1783302651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f8109370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009900 tx=0x7f22e002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 -- 172.21.15.19:0/1783302651 shutdown_connections 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 --2- 172.21.15.19:0/1783302651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f81098b0 0x7f22f810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 --2- 172.21.15.19:0/1783302651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f8109370 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 --2- 172.21.15.19:0/1783302651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f81045f0 0x7f22f8106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 -- 172.21.15.19:0/1783302651 >> 172.21.15.19:0/1783302651 conn(0x7f22f80fdbf0 msgr2=0x7f22f81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 -- 172.21.15.19:0/1783302651 shutdown_connections 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.664+0000 7f22fcb82640 1 -- 172.21.15.19:0/1783302651 wait complete. 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 Processor -- start 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 -- start start 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f81045f0 0x7f22f819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f81098b0 0x7f22f81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810f610 con 0x7f22f8106f50 2024-04-25T10:54:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810f490 con 0x7f22f81098b0 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.665+0000 7f22fcb82640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810f790 con 0x7f22f81045f0 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44120/0 (socket says 172.21.15.19:44120) 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 -- 172.21.15.19:0/825334730 learned_addr learned my addr 172.21.15.19:0/825334730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22f659c640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f81045f0 0x7f22f819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22f6d9d640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f81098b0 0x7f22f81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 -- 172.21.15.19:0/825334730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f81045f0 msgr2=0x7f22f819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f81045f0 0x7f22f819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 -- 172.21.15.19:0/825334730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f81098b0 msgr2=0x7f22f81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f81098b0 0x7f22f81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 -- 172.21.15.19:0/825334730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f81a1cd0 con 0x7f22f8106f50 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22effff640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f819a780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006010 tx=0x7f22e0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22ef7fe640 1 -- 172.21.15.19:0/825334730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e003f030 con 0x7f22f8106f50 2024-04-25T10:54:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22ef7fe640 1 -- 172.21.15.19:0/825334730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e0003cb0 con 0x7f22f8106f50 2024-04-25T10:54:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f81a1f60 con 0x7f22f8106f50 2024-04-25T10:54:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f81a2370 con 0x7f22f8106f50 2024-04-25T10:54:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.666+0000 7f22ef7fe640 1 -- 172.21.15.19:0/825334730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0035070 con 0x7f22f8106f50 2024-04-25T10:54:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.668+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d80052c0 con 0x7f22f8106f50 2024-04-25T10:54:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.669+0000 7f22ef7fe640 1 -- 172.21.15.19:0/825334730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f22e0042030 con 0x7f22f8106f50 2024-04-25T10:54:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.669+0000 7f22ef7fe640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f22e40afaf0 0x7f22e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.670+0000 7f22f659c640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f22e40afaf0 0x7f22e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.670+0000 7f22ef7fe640 1 -- 172.21.15.19:0/825334730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f22e00ff1a0 con 0x7f22f8106f50 2024-04-25T10:54:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.670+0000 7f22f659c640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f22e40afaf0 0x7f22e40b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f22f80757f0 tx=0x7f22e800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.674+0000 7f22ef7fe640 1 -- 172.21.15.19:0/825334730 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22e0003e60 con 0x7f22f8106f50 2024-04-25T10:54:20.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:20 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1225185729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:20.772 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:20 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/603289335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:20.773 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:20 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1455279350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:20 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1225185729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:20 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/603289335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:20.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:20 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1455279350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:20.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.860+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22d8005dc0 con 0x7f22f8106f50 2024-04-25T10:54:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.861+0000 7f22ef7fe640 1 -- 172.21.15.19:0/825334730 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f22e00c1e20 con 0x7f22f8106f50 2024-04-25T10:54:20.862 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:54:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f22e40afaf0 msgr2=0x7f22e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f22e40afaf0 0x7f22e40b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f22f80757f0 tx=0x7f22e800a2d0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 msgr2=0x7f22f819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f819a780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006010 tx=0x7f22e0004340 comp rx=0 tx=0).stop 2024-04-25T10:54:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 shutdown_connections 2024-04-25T10:54:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f81098b0 0x7f22f81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f22e40afaf0 0x7f22e40b2000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22f8106f50 0x7f22f819a780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 --2- 172.21.15.19:0/825334730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f81045f0 0x7f22f819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 >> 172.21.15.19:0/825334730 conn(0x7f22f80fdbf0 msgr2=0x7f22f81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:20.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 shutdown_connections 2024-04-25T10:54:20.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.863+0000 7f22fcb82640 1 -- 172.21.15.19:0/825334730 wait complete. 2024-04-25T10:54:20.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:54:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 -- 172.21.15.19:0/2291666065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c109750 msgr2=0x7ff34c10bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 --2- 172.21.15.19:0/2291666065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c109750 0x7ff34c10bc60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff344009ca0 tx=0x7ff34402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 -- 172.21.15.19:0/2291666065 shutdown_connections 2024-04-25T10:54:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 --2- 172.21.15.19:0/2291666065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c109750 0x7ff34c10bc60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 --2- 172.21.15.19:0/2291666065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c106be0 0x7ff34c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 --2- 172.21.15.19:0/2291666065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff34c1041f0 0x7ff34c106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 -- 172.21.15.19:0/2291666065 >> 172.21.15.19:0/2291666065 conn(0x7ff34c0fdbd0 msgr2=0x7ff34c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 -- 172.21.15.19:0/2291666065 shutdown_connections 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.986+0000 7ff352020640 1 -- 172.21.15.19:0/2291666065 wait complete. 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 Processor -- start 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 -- start start 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c1041f0 0x7ff34c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c106be0 0x7ff34c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff34c109750 0x7ff34c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10f5a0 con 0x7ff34c1041f0 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10f420 con 0x7ff34c109750 2024-04-25T10:54:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff352020640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10f720 con 0x7ff34c106be0 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.987+0000 7ff34b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c1041f0 0x7ff34c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c1041f0 0x7ff34c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44134/0 (socket says 172.21.15.19:44134) 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 -- 172.21.15.19:0/3976486316 learned_addr learned my addr 172.21.15.19:0/3976486316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 -- 172.21.15.19:0/3976486316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c106be0 msgr2=0x7ff34c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34affd640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c106be0 0x7ff34c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34bfff640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff34c109750 0x7ff34c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c106be0 0x7ff34c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 -- 172.21.15.19:0/3976486316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff34c109750 msgr2=0x7ff34c1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff34c109750 0x7ff34c1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 -- 172.21.15.19:0/3976486316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c1a1be0 con 0x7ff34c1041f0 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34affd640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c106be0 0x7ff34c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34bfff640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff34c109750 0x7ff34c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff34b7fe640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c1041f0 0x7ff34c19a150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c970 tx=0x7ff33c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff348ff9640 1 -- 172.21.15.19:0/3976486316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c007c10 con 0x7ff34c1041f0 2024-04-25T10:54:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff348ff9640 1 -- 172.21.15.19:0/3976486316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff33c007db0 con 0x7ff34c1041f0 2024-04-25T10:54:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c1a1ed0 con 0x7ff34c1041f0 2024-04-25T10:54:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff348ff9640 1 -- 172.21.15.19:0/3976486316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00f870 con 0x7ff34c1041f0 2024-04-25T10:54:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.988+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a2330 con 0x7ff34c1041f0 2024-04-25T10:54:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.989+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c06afa0 con 0x7ff34c1041f0 2024-04-25T10:54:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.991+0000 7ff348ff9640 1 -- 172.21.15.19:0/3976486316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff33c00fa10 con 0x7ff34c1041f0 2024-04-25T10:54:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.991+0000 7ff348ff9640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3340afaf0 0x7ff3340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.991+0000 7ff34affd640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3340afaf0 0x7ff3340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.992+0000 7ff348ff9640 1 -- 172.21.15.19:0/3976486316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7ff33c0df260 con 0x7ff34c1041f0 2024-04-25T10:54:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.992+0000 7ff34affd640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3340afaf0 0x7ff3340b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff3400058c0 tx=0x7ff340005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:20.995+0000 7ff348ff9640 1 -- 172.21.15.19:0/3976486316 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff33c0a7f70 con 0x7ff34c1041f0 2024-04-25T10:54:21.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.222+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff34c02dc50 con 0x7ff34c1041f0 2024-04-25T10:54:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.223+0000 7ff348ff9640 1 -- 172.21.15.19:0/3976486316 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7ff33c0a7870 con 0x7ff34c1041f0 2024-04-25T10:54:21.223 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:21.224 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:54:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:54:21.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3340afaf0 msgr2=0x7ff3340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3340afaf0 0x7ff3340b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff3400058c0 tx=0x7ff340005d50 comp rx=0 tx=0).stop 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c1041f0 msgr2=0x7ff34c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c1041f0 0x7ff34c19a150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c970 tx=0x7ff33c00ce30 comp rx=0 tx=0).stop 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 shutdown_connections 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff34c109750 0x7ff34c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c106be0 0x7ff34c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff3340afaf0 0x7ff3340b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 --2- 172.21.15.19:0/3976486316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c1041f0 0x7ff34c19a150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 >> 172.21.15.19:0/3976486316 conn(0x7ff34c0fdbd0 msgr2=0x7ff34c104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 shutdown_connections 2024-04-25T10:54:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.225+0000 7ff352020640 1 -- 172.21.15.19:0/3976486316 wait complete. 2024-04-25T10:54:21.237 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2824968410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec104d10 msgr2=0x7ff2ec105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2824968410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec104d10 0x7ff2ec105120 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009900 tx=0x7ff2dc02a410 comp rx=0 tx=0).stop 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2824968410 shutdown_connections 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2824968410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2ec109a10 0x7ff2ec10be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2824968410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec104d10 0x7ff2ec105120 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2824968410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2ec103ad0 0x7ff2ec1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2824968410 >> 172.21.15.19:0/2824968410 conn(0x7ff2ec0fecc0 msgr2=0x7ff2ec101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2824968410 shutdown_connections 2024-04-25T10:54:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2824968410 wait complete. 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.350+0000 7ff2f1eb6640 1 Processor -- start 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2f1eb6640 1 -- start start 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2f1eb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2ec103ad0 0x7ff2ec0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2f1eb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2ec104d10 0x7ff2ec074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2f1eb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec109a10 0x7ff2ec075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2f1eb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10f5d0 con 0x7ff2ec109a10 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2f1eb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10f450 con 0x7ff2ec104d10 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2f1eb6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10f750 con 0x7ff2ec103ad0 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec109a10 0x7ff2ec075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec109a10 0x7ff2ec075430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44152/0 (socket says 172.21.15.19:44152) 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 -- 172.21.15.19:0/2303793631 learned_addr learned my addr 172.21.15.19:0/2303793631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2eaffd640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2ec104d10 0x7ff2ec074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2eb7fe640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2ec103ad0 0x7ff2ec0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 -- 172.21.15.19:0/2303793631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2ec103ad0 msgr2=0x7ff2ec0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2ec103ad0 0x7ff2ec0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 -- 172.21.15.19:0/2303793631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2ec104d10 msgr2=0x7ff2ec074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2ec104d10 0x7ff2ec074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2ebfff640 1 -- 172.21.15.19:0/2303793631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec075b60 con 0x7ff2ec109a10 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.351+0000 7ff2eaffd640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2ec104d10 0x7ff2ec074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.352+0000 7ff2ebfff640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec109a10 0x7ff2ec075430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400eaa0 tx=0x7ff2e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.352+0000 7ff2e8ff9640 1 -- 172.21.15.19:0/2303793631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e400cdb0 con 0x7ff2ec109a10 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.352+0000 7ff2e8ff9640 1 -- 172.21.15.19:0/2303793631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e4005020 con 0x7ff2ec109a10 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.352+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec1a4320 con 0x7ff2ec109a10 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.352+0000 7ff2e8ff9640 1 -- 172.21.15.19:0/2303793631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40116d0 con 0x7ff2ec109a10 2024-04-25T10:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.352+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec1a4730 con 0x7ff2ec109a10 2024-04-25T10:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.353+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ec069310 con 0x7ff2ec109a10 2024-04-25T10:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.354+0000 7ff2e8ff9640 1 -- 172.21.15.19:0/2303793631 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff2e40051d0 con 0x7ff2ec109a10 2024-04-25T10:54:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.355+0000 7ff2e8ff9640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff2d80afaf0 0x7ff2d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.355+0000 7ff2eb7fe640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff2d80afaf0 0x7ff2d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.356+0000 7ff2e8ff9640 1 -- 172.21.15.19:0/2303793631 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7ff2e40dee10 con 0x7ff2ec109a10 2024-04-25T10:54:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.356+0000 7ff2eb7fe640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff2d80afaf0 0x7ff2d80b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005d00 tx=0x7ff2d4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.360+0000 7ff2e8ff9640 1 -- 172.21.15.19:0/2303793631 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2e40e2030 con 0x7ff2ec109a10 2024-04-25T10:54:21.515 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:21 smithi019 ceph-mon[29209]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:21.515 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:21 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2219626543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:21.515 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:21 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/825334730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:21.515 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:21 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3976486316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.549+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2ec0637a0 con 0x7ff2ec109a10 2024-04-25T10:54:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.551+0000 7ff2e8ff9640 1 -- 172.21.15.19:0/2303793631 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7ff2e4016030 con 0x7ff2ec109a10 2024-04-25T10:54:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:21.552 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:54:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.553+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff2d80afaf0 msgr2=0x7ff2d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.553+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff2d80afaf0 0x7ff2d80b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005d00 tx=0x7ff2d4005c50 comp rx=0 tx=0).stop 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.553+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec109a10 msgr2=0x7ff2ec075430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.553+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec109a10 0x7ff2ec075430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400eaa0 tx=0x7ff2e400ef60 comp rx=0 tx=0).stop 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.553+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 shutdown_connections 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.554+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff2d80afaf0 0x7ff2d80b2000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.554+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2ec109a10 0x7ff2ec075430 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.554+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2ec104d10 0x7ff2ec074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.554+0000 7ff2f1eb6640 1 --2- 172.21.15.19:0/2303793631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2ec103ad0 0x7ff2ec0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.554+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 >> 172.21.15.19:0/2303793631 conn(0x7ff2ec0fecc0 msgr2=0x7ff2ec1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.554+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 shutdown_connections 2024-04-25T10:54:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.554+0000 7ff2f1eb6640 1 -- 172.21.15.19:0/2303793631 wait complete. 2024-04-25T10:54:21.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:52:32.071033+0000', 'modified': '2024-04-25T10:52:36.113733+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6839/386384439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 386384439}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 386384439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/850740995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 850740995}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 850740995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24533, 'qdb_cluster': [24533]} 2024-04-25T10:54:21.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:54:21.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:54:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 -- 172.21.15.19:0/4257080862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 msgr2=0x7fdb4c1037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 --2- 172.21.15.19:0/4257080862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c1037c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009900 tx=0x7fdb4802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 -- 172.21.15.19:0/4257080862 shutdown_connections 2024-04-25T10:54:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 --2- 172.21.15.19:0/4257080862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1080b0 0x7fdb4c10a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 --2- 172.21.15.19:0/4257080862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c1037c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 --2- 172.21.15.19:0/4257080862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb4c102170 0x7fdb4c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 -- 172.21.15.19:0/4257080862 >> 172.21.15.19:0/4257080862 conn(0x7fdb4c0fd340 msgr2=0x7fdb4c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 -- 172.21.15.19:0/4257080862 shutdown_connections 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.683+0000 7fdb542bf640 1 -- 172.21.15.19:0/4257080862 wait complete. 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 Processor -- start 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 -- start start 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c102170 0x7fdb4c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb4c1080b0 0x7fdb4c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c10d400 con 0x7fdb4c1033b0 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c10d280 con 0x7fdb4c1080b0 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb542bf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c10d580 con 0x7fdb4c102170 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44168/0 (socket says 172.21.15.19:44168) 2024-04-25T10:54:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb5185a640 1 -- 172.21.15.19:0/93811326 learned_addr learned my addr 172.21.15.19:0/93811326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb5205b640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c102170 0x7fdb4c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.684+0000 7fdb5285c640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb4c1080b0 0x7fdb4c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb5185a640 1 -- 172.21.15.19:0/93811326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c102170 msgr2=0x7fdb4c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb5185a640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c102170 0x7fdb4c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb5185a640 1 -- 172.21.15.19:0/93811326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb4c1080b0 msgr2=0x7fdb4c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb5185a640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb4c1080b0 0x7fdb4c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb5185a640 1 -- 172.21.15.19:0/93811326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c1a1d50 con 0x7fdb4c1033b0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb5205b640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c102170 0x7fdb4c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb5185a640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c19a750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdb48006010 tx=0x7fdb48004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb437fe640 1 -- 172.21.15.19:0/93811326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4803f030 con 0x7fdb4c1033b0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb437fe640 1 -- 172.21.15.19:0/93811326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb48003cb0 con 0x7fdb4c1033b0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c1a1fe0 con 0x7fdb4c1033b0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb437fe640 1 -- 172.21.15.19:0/93811326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48035070 con 0x7fdb4c1033b0 2024-04-25T10:54:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.685+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c1a2440 con 0x7fdb4c1033b0 2024-04-25T10:54:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb437fe640 1 -- 172.21.15.19:0/93811326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdb48042030 con 0x7fdb4c1033b0 2024-04-25T10:54:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 Processor -- start 2024-04-25T10:54:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 -- start start 2024-04-25T10:54:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd6c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd6cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd73c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56324dccd740 con 0x56324d99e3b0 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56324dc693a0 con 0x56324d9c7270 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5b170740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56324dcc9700 con 0x56324d9a6590 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5205b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd6cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd73c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5205b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd6cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44170/0 (socket says 172.21.15.19:44170) 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd73c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:45758/0 (socket says 172.21.15.19:45758) 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.688+0000 7fdb5205b640 1 -- 172.21.15.19:0/3874221719 learned_addr learned my addr 172.21.15.19:0/3874221719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5285c640 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd6c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb437fe640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fdb380afb70 0x7fdb380b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5205b640 1 -- 172.21.15.19:0/3874221719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 msgr2=0x56324dd73c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5205b640 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd73c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5205b640 1 -- 172.21.15.19:0/3874221719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 msgr2=0x56324dd6c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5205b640 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd6c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5205b640 1 -- 172.21.15.19:0/3874221719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324dc85750 con 0x56324d99e3b0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5285c640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fdb380afb70 0x7fdb380b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb437fe640 1 -- 172.21.15.19:0/93811326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdb480ff2c0 con 0x7fdb4c1033b0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb5205b640 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd6cd00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00cc40 tx=0x7fdb3c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8de933f35b2879e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb40bfe640 1 -- 172.21.15.19:0/3874221719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c014070 con 0x56324d99e3b0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb40bfe640 1 -- 172.21.15.19:0/3874221719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb3c019b10 con 0x56324d99e3b0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.689+0000 7fdb40bfe640 1 -- 172.21.15.19:0/3874221719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c010be0 con 0x56324d99e3b0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 -- 172.21.15.19:0/3874221719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 msgr2=0x56324dd6cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd6cd00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00cc40 tx=0x7fdb3c00e720 comp rx=0 tx=0).stop 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5285c640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fdb380afb70 0x7fdb380b2080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56324dd6d950 tx=0x7fdb44004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 -- 172.21.15.19:0/3874221719 shutdown_connections 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd73c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd6cd00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 --2- 172.21.15.19:0/3874221719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd6c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 -- 172.21.15.19:0/3874221719 >> 172.21.15.19:0/3874221719 conn(0x56324d9bb3c0 msgr2=0x56324dd63d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 -- 172.21.15.19:0/3874221719 shutdown_connections 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.690+0000 7fdb5b170740 1 -- 172.21.15.19:0/3874221719 wait complete. 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 -- start start 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 Processor -- start 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d99e3b0 0x56324dd76650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d9a6590 0x56324dd76b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd7daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56324dd74650 con 0x56324d9a6590 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56324dc54fe0 con 0x56324d9c7270 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56324dd747d0 con 0x56324d99e3b0 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5b170740 10 client.0 _ll_register_callbacks cb 0x56324dc6b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d9a6590 0x56324dd76b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d9a6590 0x56324dd76b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44176/0 (socket says 172.21.15.19:44176) 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5185a640 1 -- 172.21.15.19:0/1945428309 learned_addr learned my addr 172.21.15.19:0/1945428309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:21.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5205b640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d99e3b0 0x56324dd76650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.691+0000 7fdb5285c640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd7daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5185a640 1 -- 172.21.15.19:0/1945428309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d99e3b0 msgr2=0x56324dd76650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5185a640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d99e3b0 0x56324dd76650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5185a640 1 -- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 msgr2=0x56324dd7daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5185a640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd7daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5185a640 1 -- 172.21.15.19:0/1945428309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324d9df2d0 con 0x56324d9a6590 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5285c640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd7daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5185a640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d9a6590 0x56324dd76b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdb480fcdd0 tx=0x7fdb480c3ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df9b44571701a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb35ffb640 1 -- 172.21.15.19:0/1945428309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48049070 con 0x56324d9a6590 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb35ffb640 1 -- 172.21.15.19:0/1945428309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdb480ea0b0 con 0x56324d9a6590 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3c019b10 con 0x56324d9a6590 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb35ffb640 1 -- 172.21.15.19:0/1945428309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb480c3670 con 0x56324d9a6590 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5b170740 10 client.15180 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56324daa96b0 con 0x56324d9a6590 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb35ffb640 1 -- 172.21.15.19:0/1945428309 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb3c019b10 con 0x56324d9a6590 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.692+0000 7fdb35ffb640 1 -- 172.21.15.19:0/1945428309 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fdb4803bbd0 con 0x56324d9a6590 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5b170740 10 client.15180 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5b170740 10 client.15180 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5b170740 20 client.15180 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5b170740 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d985e40 0x56324dd84320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5b170740 4 client.15180 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56324dcdb350 con 0x56324d985e40 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5205b640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d985e40 0x56324dd84320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb5205b640 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d985e40 0x56324dd84320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d5d599c82106b5 server_cookie=dd46f90d47f959de in_seq=0 out_seq=0 2024-04-25T10:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.693+0000 7fdb35ffb640 10 client.15180 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb35ffb640 1 -- 172.21.15.19:0/1945428309 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56324dcdb350 con 0x56324d985e40 2024-04-25T10:54:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb35ffb640 10 client.15180 handle_command_reply: tid=0 2024-04-25T10:54:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 client.15180 shutdown 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d9a6590 msgr2=0x56324dd76b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d9a6590 0x56324dd76b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdb480fcdd0 tx=0x7fdb480c3ca0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 shutdown_connections 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d985e40 0x56324dd84320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d9c7270 0x56324dd7daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb35ffb640 0 client.15180 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d9a6590 0x56324dd76b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 --2- 172.21.15.19:0/1945428309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d99e3b0 0x56324dd76650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.694+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 >> 172.21.15.19:0/1945428309 conn(0x56324d9bb3c0 msgr2=0x56324dd62d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.695+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 shutdown_connections 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.695+0000 7fdb5b170740 1 -- 172.21.15.19:0/1945428309 wait complete. 2024-04-25T10:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.695+0000 7fdb5b170740 20 client.15180 trim_cache size 0 max 16384 2024-04-25T10:54:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 Processor -- start 2024-04-25T10:54:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 -- start start 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d985e40 0x56324dd6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56324dd82660 con 0x56324d99e3b0 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56324dd82d30 con 0x56324d985e40 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5b170740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56324dd83400 con 0x56324d9a6590 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd74c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44186/0 (socket says 172.21.15.19:44186) 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 -- 172.21.15.19:0/4214367282 learned_addr learned my addr 172.21.15.19:0/4214367282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5205b640 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d985e40 0x56324dd6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 -- 172.21.15.19:0/4214367282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 msgr2=0x56324dd751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 -- 172.21.15.19:0/4214367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d985e40 msgr2=0x56324dd6fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d985e40 0x56324dd6fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.714+0000 7fdb5185a640 1 -- 172.21.15.19:0/4214367282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324dd6d980 con 0x56324d99e3b0 2024-04-25T10:54:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.715+0000 7fdb5185a640 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd74c90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb480c77b0 tx=0x7fdb48009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e818298377f77c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.715+0000 7fdb37fff640 1 -- 172.21.15.19:0/4214367282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48049070 con 0x56324d99e3b0 2024-04-25T10:54:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.715+0000 7fdb37fff640 1 -- 172.21.15.19:0/4214367282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4803b460 con 0x56324d99e3b0 2024-04-25T10:54:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.715+0000 7fdb37fff640 1 -- 172.21.15.19:0/4214367282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480c6f20 con 0x56324d99e3b0 2024-04-25T10:54:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- 172.21.15.19:0/4214367282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 msgr2=0x56324dd74c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd74c90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb480c77b0 tx=0x7fdb48009620 comp rx=0 tx=0).stop 2024-04-25T10:54:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- 172.21.15.19:0/4214367282 shutdown_connections 2024-04-25T10:54:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d99e3b0 0x56324dd74c90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 --2- 172.21.15.19:0/4214367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d985e40 0x56324dd6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- 172.21.15.19:0/4214367282 >> 172.21.15.19:0/4214367282 conn(0x56324d9bb3c0 msgr2=0x56324dd70ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- 172.21.15.19:0/4214367282 shutdown_connections 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- 172.21.15.19:0/4214367282 wait complete. 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- start start 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 Processor -- start 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d985e40 0x56324dd79620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d99e3b0 0x56324dd79b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd7caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56324dd841a0 con 0x56324d985e40 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56324dd83ad0 con 0x56324d99e3b0 2024-04-25T10:54:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56324dd84870 con 0x56324d9a6590 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5b170740 10 client.0 _ll_register_callbacks cb 0x7fdb48005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5205b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d985e40 0x56324dd79620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5205b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d985e40 0x56324dd79620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44190/0 (socket says 172.21.15.19:44190) 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.716+0000 7fdb5205b640 1 -- 172.21.15.19:0/2021630728 learned_addr learned my addr 172.21.15.19:0/2021630728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5285c640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd7caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5185a640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d99e3b0 0x56324dd79b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5205b640 1 -- 172.21.15.19:0/2021630728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 msgr2=0x56324dd7caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5205b640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd7caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5205b640 1 -- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d99e3b0 msgr2=0x56324dd79b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5205b640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d99e3b0 0x56324dd79b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5205b640 1 -- 172.21.15.19:0/2021630728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324dd7d060 con 0x56324d985e40 2024-04-25T10:54:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5285c640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd7caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5205b640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d985e40 0x56324dd79620 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00cb60 tx=0x7fdb3c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee95445dd925a409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb35ffb640 1 -- 172.21.15.19:0/2021630728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c014040 con 0x56324d985e40 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb35ffb640 1 -- 172.21.15.19:0/2021630728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdb3c005510 con 0x56324d985e40 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56324dcd51c0 con 0x56324d985e40 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb35ffb640 1 -- 172.21.15.19:0/2021630728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb3c019460 con 0x56324d985e40 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5b170740 10 client.15186 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56324dd7e340 con 0x56324d985e40 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.717+0000 7fdb35ffb640 1 -- 172.21.15.19:0/2021630728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56324dcd51c0 con 0x56324d985e40 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb35ffb640 1 -- 172.21.15.19:0/2021630728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fdb3c004b70 con 0x56324d985e40 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5b170740 10 client.15186 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5b170740 10 client.15186 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5b170740 20 client.15186 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5b170740 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d9c7270 0x56324ddf0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5b170740 4 client.15186 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "status"}) -- 0x56324dc509a0 con 0x56324d9c7270 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5185a640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d9c7270 0x56324ddf0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb35ffb640 10 client.15186 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.718+0000 7fdb5185a640 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d9c7270 0x56324ddf0210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbedc31e1d962257 server_cookie=a3edcfbf140f7f8d in_seq=0 out_seq=0 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb35ffb640 1 -- 172.21.15.19:0/2021630728 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56324dc509a0 con 0x56324d9c7270 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb35ffb640 10 client.15186 handle_command_reply: tid=0 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 client.15186 shutdown 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d985e40 msgr2=0x56324dd79620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d985e40 0x56324dd79620 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00cb60 tx=0x7fdb3c005f30 comp rx=0 tx=0).stop 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 shutdown_connections 2024-04-25T10:54:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56324d9c7270 0x56324ddf0210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56324d9a6590 0x56324dd7caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56324d99e3b0 0x56324dd79b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb35ffb640 0 client.15186 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 --2- 172.21.15.19:0/2021630728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324d985e40 0x56324dd79620 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 >> 172.21.15.19:0/2021630728 conn(0x56324d9bb3c0 msgr2=0x56324dd6fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 shutdown_connections 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 1 -- 172.21.15.19:0/2021630728 wait complete. 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb5b170740 20 client.15186 trim_cache size 0 max 16384 2024-04-25T10:54:21.726 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9e90b6fe-02f0-11ef-bc93-c7b262605968", 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24533, 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 109.011549606, 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 20, 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 79, 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 114.738883933 2024-04-25T10:54:21.727 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fdb380afb70 msgr2=0x7fdb380b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb542bf640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fdb380afb70 0x7fdb380b2080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56324dd6d950 tx=0x7fdb44004a40 comp rx=0 tx=0).stop 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 msgr2=0x7fdb4c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.719+0000 7fdb542bf640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c19a750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdb48006010 tx=0x7fdb48004340 comp rx=0 tx=0).stop 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 shutdown_connections 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb4c1080b0 0x7fdb4c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fdb380afb70 0x7fdb380b2080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c1033b0 0x7fdb4c19a750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 --2- 172.21.15.19:0/93811326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c102170 0x7fdb4c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 >> 172.21.15.19:0/93811326 conn(0x7fdb4c0fd340 msgr2=0x7fdb4c104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 shutdown_connections 2024-04-25T10:54:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.720+0000 7fdb542bf640 1 -- 172.21.15.19:0/93811326 wait complete. 2024-04-25T10:54:21.732 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T10:54:21.743 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T10:54:21.743 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T10:54:21.743 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T10:54:21.743 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T10:54:21.744 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T10:54:21.744 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:54:21.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T10:54:21.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:21 smithi169 ceph-mon[33049]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:21.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:21 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2219626543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:21.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:21 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/825334730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:21.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:21 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3976486316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:21 smithi112 ceph-mon[32799]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:54:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:21 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2219626543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:21 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/825334730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:21.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:21 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3976486316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 -- 172.21.15.19:0/1202420219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 msgr2=0x7f2338109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 --2- 172.21.15.19:0/1202420219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f2338109120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f232c009900 tx=0x7f232c02a410 comp rx=0 tx=0).stop 2024-04-25T10:54:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 -- 172.21.15.19:0/1202420219 shutdown_connections 2024-04-25T10:54:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 --2- 172.21.15.19:0/1202420219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23381097b0 0x7f233810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 --2- 172.21.15.19:0/1202420219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f2338109120 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 --2- 172.21.15.19:0/1202420219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2338104290 0x7f23381066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 -- 172.21.15.19:0/1202420219 >> 172.21.15.19:0/1202420219 conn(0x7f23380fdc10 msgr2=0x7f23381000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:21.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 -- 172.21.15.19:0/1202420219 shutdown_connections 2024-04-25T10:54:21.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.857+0000 7f233d227640 1 -- 172.21.15.19:0/1202420219 wait complete. 2024-04-25T10:54:21.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 Processor -- start 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 -- start start 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2338104290 0x7f233819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f233819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23381097b0 0x7f23381a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f233810f640 con 0x7f2338106c80 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f233810f4c0 con 0x7f2338104290 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233d227640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f233810f7c0 con 0x7f23381097b0 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f233819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f233819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44214/0 (socket says 172.21.15.19:44214) 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.858+0000 7f233659c640 1 -- 172.21.15.19:0/2509223471 learned_addr learned my addr 172.21.15.19:0/2509223471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f2336d9d640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2338104290 0x7f233819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233759e640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23381097b0 0x7f23381a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233659c640 1 -- 172.21.15.19:0/2509223471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23381097b0 msgr2=0x7f23381a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233659c640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23381097b0 0x7f23381a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233659c640 1 -- 172.21.15.19:0/2509223471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2338104290 msgr2=0x7f233819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233659c640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2338104290 0x7f233819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233659c640 1 -- 172.21.15.19:0/2509223471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23381a1d80 con 0x7f2338106c80 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f2336d9d640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2338104290 0x7f233819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233659c640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f233819a780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f232c006010 tx=0x7f232c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f231ffff640 1 -- 172.21.15.19:0/2509223471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c03f030 con 0x7f2338106c80 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23381a2010 con 0x7f2338106c80 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f231ffff640 1 -- 172.21.15.19:0/2509223471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f232c003cb0 con 0x7f2338106c80 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f231ffff640 1 -- 172.21.15.19:0/2509223471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c035070 con 0x7f2338106c80 2024-04-25T10:54:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.859+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23381a2500 con 0x7f2338106c80 2024-04-25T10:54:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.860+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23380ff730 con 0x7f2338106c80 2024-04-25T10:54:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.862+0000 7f231ffff640 1 -- 172.21.15.19:0/2509223471 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f232c042030 con 0x7f2338106c80 2024-04-25T10:54:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.862+0000 7f231ffff640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f23200afae0 0x7f23200b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.862+0000 7f2336d9d640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f23200afae0 0x7f23200b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.862+0000 7f231ffff640 1 -- 172.21.15.19:0/2509223471 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f232c0ff320 con 0x7f2338106c80 2024-04-25T10:54:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.863+0000 7f2336d9d640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f23200afae0 0x7f23200b1ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f23300058c0 tx=0x7f2330005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:21.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:21.867+0000 7f231ffff640 1 -- 172.21.15.19:0/2509223471 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f232c003e60 con 0x7f2338106c80 2024-04-25T10:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.109+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f23380637a0 con 0x7f2338106c80 2024-04-25T10:54:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.346+0000 7f231ffff640 1 -- 172.21.15.19:0/2509223471 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f232c0c7f10 con 0x7f2338106c80 2024-04-25T10:54:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f23200afae0 msgr2=0x7f23200b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f23200afae0 0x7f23200b1ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f23300058c0 tx=0x7f2330005850 comp rx=0 tx=0).stop 2024-04-25T10:54:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 msgr2=0x7f233819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f233819a780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f232c006010 tx=0x7f232c004340 comp rx=0 tx=0).stop 2024-04-25T10:54:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 shutdown_connections 2024-04-25T10:54:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23381097b0 0x7f23381a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f23200afae0 0x7f23200b1ff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2338106c80 0x7f233819a780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 --2- 172.21.15.19:0/2509223471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2338104290 0x7f233819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.350+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 >> 172.21.15.19:0/2509223471 conn(0x7f23380fdc10 msgr2=0x7f2338104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.352+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 shutdown_connections 2024-04-25T10:54:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:22.353+0000 7f233d227640 1 -- 172.21.15.19:0/2509223471 wait complete. 2024-04-25T10:54:22.374 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:54:22.398 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:54:22.398 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T10:54:22.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:22.399 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:54:22.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:22.416 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T10:54:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T10:54:22.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:54:22.487 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T10:54:22.487 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:54:22.487 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:22.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T10:54:22.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:22 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2303793631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:22.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:22 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2509223471' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T10:54:22.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setuser ceph since I am not root 2024-04-25T10:54:22.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setgroup ceph since I am not root 2024-04-25T10:54:22.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:22.569+0000 7f48be647200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:54:22.735 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2303793631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:22.735 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2509223471' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T10:54:22.736 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: ignoring --setuser ceph since I am not root 2024-04-25T10:54:22.736 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: ignoring --setgroup ceph since I am not root 2024-04-25T10:54:22.736 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-mgr[34184]: -- 172.21.15.112:0/2903601759 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e2b6dc5a0 con 0x557e30466480 2024-04-25T10:54:22.736 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:22.548+0000 7f1c8cc22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:54:22.736 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:22.734+0000 7f1c8cc22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:54:22.737 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2303793631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:22.737 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2509223471' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T10:54:22.737 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: ignoring --setuser ceph since I am not root 2024-04-25T10:54:22.737 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: ignoring --setgroup ceph since I am not root 2024-04-25T10:54:22.737 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-mgr[34909]: -- 172.21.15.169:0/3383036209 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561141045a0 con 0x556118e8e900 2024-04-25T10:54:22.737 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:22.548+0000 7f85b9a22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:54:22.737 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:22.735+0000 7f85b9a22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:54:22.794 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:54:23.001 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:22.841+0000 7f85b9a22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:54:23.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:22.788+0000 7f48be647200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:54:23.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:22.902+0000 7f48be647200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:54:23.058 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:22.838+0000 7f1c8cc22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:54:23.325 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:23.233+0000 7f48be647200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:54:23.325 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:23.324+0000 7f48be647200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:54:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:23 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2509223471' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T10:54:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:23 smithi169 ceph-mon[33049]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T10:54:23.501 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:23.153+0000 7f85b9a22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:54:23.501 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:23.239+0000 7f85b9a22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:54:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:23 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2509223471' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T10:54:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:23 smithi112 ceph-mon[32799]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T10:54:23.559 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:23.145+0000 7f1c8cc22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:54:23.559 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:23.230+0000 7f1c8cc22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:54:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:23 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2509223471' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T10:54:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:23 smithi019 ceph-mon[29209]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T10:54:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 -- 172.21.15.19:0/3053560409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f643010e7d0 msgr2=0x7f6430110cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 --2- 172.21.15.19:0/3053560409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f643010e7d0 0x7f6430110cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6428009f90 tx=0x7f642800b330 comp rx=0 tx=0).stop 2024-04-25T10:54:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 -- 172.21.15.19:0/3053560409 shutdown_connections 2024-04-25T10:54:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 --2- 172.21.15.19:0/3053560409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f643010e7d0 0x7f6430110cb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 --2- 172.21.15.19:0/3053560409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105360 0x7f6430105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 --2- 172.21.15.19:0/3053560409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6430105cb0 0x7f643010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 -- 172.21.15.19:0/3053560409 >> 172.21.15.19:0/3053560409 conn(0x7f64300fedb0 msgr2=0x7f64301011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 -- 172.21.15.19:0/3053560409 shutdown_connections 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 -- 172.21.15.19:0/3053560409 wait complete. 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.516+0000 7f6435f88700 1 Processor -- start 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f6435f88700 1 -- start start 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f6435f88700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6430105360 0x7f6430103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f6435f88700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105cb0 0x7f6430103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f6435f88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f643010e7d0 0x7f6430103ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f6435f88700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6430114380 con 0x7f6430105cb0 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f6435f88700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6430114200 con 0x7f643010e7d0 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f6435f88700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6430114500 con 0x7f6430105360 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f642effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105cb0 0x7f6430103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f642effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105cb0 0x7f6430103590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43204/0 (socket says 172.21.15.19:43204) 2024-04-25T10:54:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f642f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6430105360 0x7f6430103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f642effd700 1 -- 172.21.15.19:0/323863101 learned_addr learned my addr 172.21.15.19:0/323863101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.517+0000 7f642ffff700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f643010e7d0 0x7f6430103ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642ffff700 1 -- 172.21.15.19:0/323863101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6430105360 msgr2=0x7f6430103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642ffff700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6430105360 0x7f6430103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642ffff700 1 -- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105cb0 msgr2=0x7f6430103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642ffff700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105cb0 0x7f6430103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642ffff700 1 -- 172.21.15.19:0/323863101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64301a9030 con 0x7f643010e7d0 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642effd700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105cb0 0x7f6430103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642f7fe700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6430105360 0x7f6430103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642ffff700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f643010e7d0 0x7f6430103ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f642800b6c0 tx=0x7f642800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642cff9700 1 -- 172.21.15.19:0/323863101 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428021070 con 0x7f643010e7d0 2024-04-25T10:54:24.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.518+0000 7f642cff9700 1 -- 172.21.15.19:0/323863101 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6428007d60 con 0x7f643010e7d0 2024-04-25T10:54:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.519+0000 7f642cff9700 1 -- 172.21.15.19:0/323863101 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428005040 con 0x7f643010e7d0 2024-04-25T10:54:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.519+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64301a92c0 con 0x7f643010e7d0 2024-04-25T10:54:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.519+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64301a9770 con 0x7f643010e7d0 2024-04-25T10:54:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.520+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6430051190 con 0x7f643010e7d0 2024-04-25T10:54:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.522+0000 7f642cff9700 1 -- 172.21.15.19:0/323863101 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f6428024070 con 0x7f643010e7d0 2024-04-25T10:54:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.522+0000 7f642cff9700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 0x7f64180b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.522+0000 7f642f7fe700 1 -- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 msgr2=0x7f64180b2160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.522+0000 7f642cff9700 1 -- 172.21.15.19:0/323863101 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f64280dfc70 con 0x7f643010e7d0 2024-04-25T10:54:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.522+0000 7f642f7fe700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 0x7f64180b2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:24.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.526+0000 7f642cff9700 1 -- 172.21.15.19:0/323863101 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f64280a8890 con 0x7f643010e7d0 2024-04-25T10:54:24.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.723+0000 7f642f7fe700 1 -- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 msgr2=0x7f64180b2160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.723+0000 7f642f7fe700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 0x7f64180b2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:54:24.748 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:24.403+0000 7f1c8cc22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:54:24.748 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:24.577+0000 7f1c8cc22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:54:24.748 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:24.747+0000 7f1c8cc22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:54:24.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:24.435+0000 7f85b9a22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:54:24.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:24.614+0000 7f85b9a22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:54:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.781+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f64301a9a70 con 0x7f643010e7d0 2024-04-25T10:54:24.782 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:24.605+0000 7f48be647200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:54:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.782+0000 7f642cff9700 1 -- 172.21.15.19:0/323863101 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f64280a8190 con 0x7f643010e7d0 2024-04-25T10:54:24.783 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 msgr2=0x7f64180b2160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 0x7f64180b2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f643010e7d0 msgr2=0x7f6430103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f643010e7d0 0x7f6430103ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f642800b6c0 tx=0x7f642800cc30 comp rx=0 tx=0).stop 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 shutdown_connections 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f64180afc50 0x7f64180b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f643010e7d0 0x7f6430103ad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6430105360 0x7f6430103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 --2- 172.21.15.19:0/323863101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430105cb0 0x7f6430103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:24.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 >> 172.21.15.19:0/323863101 conn(0x7f64300fedb0 msgr2=0x7f64300ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:24.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 shutdown_connections 2024-04-25T10:54:24.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:24.784+0000 7f6435f88700 1 -- 172.21.15.19:0/323863101 wait complete. 2024-04-25T10:54:25.041 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:24 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/323863101' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T10:54:25.087 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:24 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/323863101' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T10:54:25.088 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:24.789+0000 7f85b9a22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:54:25.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/323863101' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T10:54:25.102 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:24.790+0000 7f48be647200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:54:25.103 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:24.972+0000 7f48be647200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:54:25.309 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:25.040+0000 7f1c8cc22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:54:25.309 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:25.132+0000 7f1c8cc22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:54:25.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:25.285+0000 7f48be647200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:54:25.389 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:25.086+0000 7f85b9a22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:54:25.389 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:25.181+0000 7f85b9a22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:54:25.535 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:54:25.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:25.536 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:54:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:54:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:54:25.554 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:54:25.554 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:54:25.554 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:54:25.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:54:25.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-04-25T10:54:25.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-04-25T10:54:25.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:54:25.554 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:54:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:54:25.651 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:25.384+0000 7f48be647200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:54:25.651 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:25.602+0000 7f48be647200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:54:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:25.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:25.744 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:54:25.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:25.387+0000 7f85b9a22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:54:25.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:25.586+0000 7f85b9a22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:54:25.768 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:54:25.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:25.769 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:54:25.809 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:25.335+0000 7f1c8cc22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:54:25.809 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:25.528+0000 7f1c8cc22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:54:25.833 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:54:25.833 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:54:25.833 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:54:25.833 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:54:25.834 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:54:25.834 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:54:25.834 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:54:25.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:54:25.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:54:25.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:54:25.834 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:54:25.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:54:25.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:25.824+0000 7f48be647200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:54:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:26.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:26.102 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:54:26.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:26.118 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T10:54:26.175 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:54:26.175 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:54:26.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:54:26.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:54:26.175 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:54:26.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:54:26.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:26.295 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:54:26.295 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:54:26.295 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:54:26.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:54:26.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:54:26.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:54:26.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:54:26.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:54:26.296 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:54:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:54:26.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:26.427 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:54:26.427 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:54:26.427 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:54:26.427 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:54:26.427 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:54:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:54:26.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:26.501 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:26.196+0000 7f85b9a22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:54:26.506 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:54:26.507 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:26.535 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:54:26.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:54:26.536 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:26.551 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:26.124+0000 7f1c8cc22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:54:26.591 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T10:54:26.591 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T10:54:26.591 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/ffc04145-58c1-4c1c-87dd-774d8647033f 2024-04-25T10:54:26.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:26.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/ffc04145-58c1-4c1c-87dd-774d8647033f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T10:54:26.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:26.485+0000 7f48be647200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:54:26.671 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T10:54:26.671 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T10:54:26.785 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T10:54:26.786 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T10:54:26.786 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.169:3300" 2024-04-25T10:54:26.786 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@9e90b6fe-02f0-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/ffc04145-58c1-4c1c-87dd-774d8647033f 2024-04-25T10:54:26.786 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.169:3300" will pass to kernel 2024-04-25T10:54:26.786 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:54:26.786 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:54:26.786 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:54:26.787 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:54:26.787 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:54:26.788 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:54:26.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:26.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:26.808 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:26.550+0000 7f1c8cc22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:54:26.808 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:26.648+0000 7f1c8cc22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:54:26.816 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:54:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:54:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:26.916 INFO:teuthology.orchestra.run.smithi019.stdout:client24950 2024-04-25T10:54:26.916 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:54:26.916 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:54:26.932 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:54:26.933 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:26.626+0000 7f85b9a22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:54:26.933 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:26.725+0000 7f85b9a22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:54:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:54:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:27.031 INFO:teuthology.orchestra.run.smithi019.stdout:client24950 2024-04-25T10:54:27.032 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:54:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:54:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:27.090 INFO:teuthology.orchestra.run.smithi019.stdout:client24950 2024-04-25T10:54:27.090 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968.client24950/status') 2024-04-25T10:54:27.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:54:27.151 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.24950 (3)172.21.15.19:0/1909936104 2024-04-25T10:54:27.151 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-25T10:54:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-25T10:54:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-25T10:54:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:71 bytes copied, 2.6798e-05 s, 2.6 MB/s 2024-04-25T10:54:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:54:27.159 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T10:54:27.171 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-25T10:54:27.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:54:27.216 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:26.958+0000 7f48be647200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:54:27.216 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:27.066+0000 7f48be647200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:54:27.251 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:26.930+0000 7f85b9a22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:54:27.309 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:26.853+0000 7f1c8cc22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:54:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.308+0000 7f4c315ee640 1 -- 172.21.15.19:0/3748246369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 msgr2=0x7f4c2c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.308+0000 7f4c315ee640 1 --2- 172.21.15.19:0/3748246369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c10ad40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009ca0 tx=0x7f4c2402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.308+0000 7f4c315ee640 1 -- 172.21.15.19:0/3748246369 shutdown_connections 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.308+0000 7f4c315ee640 1 --2- 172.21.15.19:0/3748246369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c10ad40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.308+0000 7f4c315ee640 1 --2- 172.21.15.19:0/3748246369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c2c103c20 0x7f4c2c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.308+0000 7f4c315ee640 1 --2- 172.21.15.19:0/3748246369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c2c1029e0 0x7f4c2c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.308+0000 7f4c315ee640 1 -- 172.21.15.19:0/3748246369 >> 172.21.15.19:0/3748246369 conn(0x7f4c2c0fdbd0 msgr2=0x7f4c2c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 -- 172.21.15.19:0/3748246369 shutdown_connections 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 -- 172.21.15.19:0/3748246369 wait complete. 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 Processor -- start 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 -- start start 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c2c1029e0 0x7f4c2c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c2c103c20 0x7f4c2c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c10e4f0 con 0x7f4c2c108920 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c10e370 con 0x7f4c2c1029e0 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.309+0000 7f4c315ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c10e670 con 0x7f4c2c103c20 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43232/0 (socket says 172.21.15.19:43232) 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c2c1029e0 0x7f4c2c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 -- 172.21.15.19:0/6334789 learned_addr learned my addr 172.21.15.19:0/6334789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c23fff640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c2c103c20 0x7f4c2c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 -- 172.21.15.19:0/6334789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c2c103c20 msgr2=0x7f4c2c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c2c103c20 0x7f4c2c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 -- 172.21.15.19:0/6334789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c2c1029e0 msgr2=0x7f4c2c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c2c1029e0 0x7f4c2c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 -- 172.21.15.19:0/6334789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c1a1d40 con 0x7f4c2c108920 2024-04-25T10:54:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c2b7fe640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c1a1520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005e30 tx=0x7f4c24031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c28ff9640 1 -- 172.21.15.19:0/6334789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24003b70 con 0x7f4c2c108920 2024-04-25T10:54:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2c1a1fd0 con 0x7f4c2c108920 2024-04-25T10:54:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c28ff9640 1 -- 172.21.15.19:0/6334789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c24003d10 con 0x7f4c2c108920 2024-04-25T10:54:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c28ff9640 1 -- 172.21.15.19:0/6334789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2402fac0 con 0x7f4c2c108920 2024-04-25T10:54:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.310+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2c1a2490 con 0x7f4c2c108920 2024-04-25T10:54:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.312+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2c107730 con 0x7f4c2c108920 2024-04-25T10:54:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.313+0000 7f4c28ff9640 1 -- 172.21.15.19:0/6334789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f4c2402fc60 con 0x7f4c2c108920 2024-04-25T10:54:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.314+0000 7f4c28ff9640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 0x7f4c180b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.314+0000 7f4c2affd640 1 -- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 msgr2=0x7f4c180b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.314+0000 7f4c28ff9640 1 -- 172.21.15.19:0/6334789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f4c24100160 con 0x7f4c2c108920 2024-04-25T10:54:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.314+0000 7f4c2affd640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 0x7f4c180b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.318+0000 7f4c28ff9640 1 -- 172.21.15.19:0/6334789 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c240c8d50 con 0x7f4c2c108920 2024-04-25T10:54:27.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.515+0000 7f4c2affd640 1 -- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 msgr2=0x7f4c180b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:27.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.515+0000 7f4c2affd640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 0x7f4c180b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:54:27.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.536+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c2c0637a0 con 0x7f4c2c108920 2024-04-25T10:54:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.537+0000 7f4c28ff9640 1 -- 172.21.15.19:0/6334789 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7f4c240c8650 con 0x7f4c2c108920 2024-04-25T10:54:27.538 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:27.539 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:54:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 msgr2=0x7f4c180b2050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 0x7f4c180b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 msgr2=0x7f4c2c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c1a1520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005e30 tx=0x7f4c24031840 comp rx=0 tx=0).stop 2024-04-25T10:54:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 shutdown_connections 2024-04-25T10:54:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f4c180afb40 0x7f4c180b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c2c108920 0x7f4c2c1a1520 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c2c103c20 0x7f4c2c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 --2- 172.21.15.19:0/6334789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c2c1029e0 0x7f4c2c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 >> 172.21.15.19:0/6334789 conn(0x7f4c2c0fdbd0 msgr2=0x7f4c2c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 shutdown_connections 2024-04-25T10:54:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.540+0000 7f4c315ee640 1 -- 172.21.15.19:0/6334789 wait complete. 2024-04-25T10:54:27.551 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:27.291+0000 7f48be647200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:54:27.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:54:27.553 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:54:27.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 -- 172.21.15.19:0/3998742733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 msgr2=0x7f8164106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 --2- 172.21.15.19:0/3998742733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f8164106a00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8160009900 tx=0x7f816002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:27.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 -- 172.21.15.19:0/3998742733 shutdown_connections 2024-04-25T10:54:27.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 --2- 172.21.15.19:0/3998742733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81641098a0 0x7f816410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 --2- 172.21.15.19:0/3998742733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8164106f40 0x7f8164109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 --2- 172.21.15.19:0/3998742733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f8164106a00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 -- 172.21.15.19:0/3998742733 >> 172.21.15.19:0/3998742733 conn(0x7f81640fdbc0 msgr2=0x7f8164100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 -- 172.21.15.19:0/3998742733 shutdown_connections 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 -- 172.21.15.19:0/3998742733 wait complete. 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.708+0000 7f816b355640 1 Processor -- start 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f816b355640 1 -- start start 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f816b355640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f816419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f816b355640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8164106f40 0x7f816419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f816b355640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81641098a0 0x7f81641a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f816b355640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f816410f5f0 con 0x7f81641045e0 2024-04-25T10:54:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f816b355640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f816410f470 con 0x7f8164106f40 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f816b355640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f816410f770 con 0x7f81641098a0 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f816419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f816419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43242/0 (socket says 172.21.15.19:43242) 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 -- 172.21.15.19:0/1786393262 learned_addr learned my addr 172.21.15.19:0/1786393262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81688f0640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8164106f40 0x7f816419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81698f2640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81641098a0 0x7f81641a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 -- 172.21.15.19:0/1786393262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81641098a0 msgr2=0x7f81641a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81641098a0 0x7f81641a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 -- 172.21.15.19:0/1786393262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8164106f40 msgr2=0x7f816419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8164106f40 0x7f816419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 -- 172.21.15.19:0/1786393262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81641a1d90 con 0x7f81641045e0 2024-04-25T10:54:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81688f0640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8164106f40 0x7f816419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.709+0000 7f81690f1640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f816419a250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8160006010 tx=0x7f8160004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.710+0000 7f815a7fc640 1 -- 172.21.15.19:0/1786393262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816003f030 con 0x7f81641045e0 2024-04-25T10:54:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.710+0000 7f815a7fc640 1 -- 172.21.15.19:0/1786393262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160031910 con 0x7f81641045e0 2024-04-25T10:54:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.710+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641a2020 con 0x7f81641045e0 2024-04-25T10:54:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.710+0000 7f815a7fc640 1 -- 172.21.15.19:0/1786393262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160035070 con 0x7f81641045e0 2024-04-25T10:54:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.710+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641a2560 con 0x7f81641045e0 2024-04-25T10:54:27.715 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:27 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/6334789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:27.715 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:27.374+0000 7f1c8cc22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:54:27.715 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:27.468+0000 7f1c8cc22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:54:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.711+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81640ff6e0 con 0x7f81641045e0 2024-04-25T10:54:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.712+0000 7f815a7fc640 1 -- 172.21.15.19:0/1786393262 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f8160042030 con 0x7f81641045e0 2024-04-25T10:54:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.713+0000 7f815a7fc640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f81500afbc0 0x7f81500b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.713+0000 7f81688f0640 1 -- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f81500afbc0 msgr2=0x7f81500b20d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.714+0000 7f815a7fc640 1 -- 172.21.15.19:0/1786393262 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f81600ff380 con 0x7f81641045e0 2024-04-25T10:54:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.714+0000 7f81688f0640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f81500afbc0 0x7f81500b20d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:27.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.717+0000 7f815a7fc640 1 -- 172.21.15.19:0/1786393262 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81600c7920 con 0x7f81641045e0 2024-04-25T10:54:27.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:27 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/6334789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:27.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:27.461+0000 7f85b9a22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:54:27.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:27.556+0000 7f85b9a22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:54:27.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:27 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/6334789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.903+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f816402dce0 con 0x7f81641045e0 2024-04-25T10:54:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.904+0000 7f815a7fc640 1 -- 172.21.15.19:0/1786393262 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f81600cbc20 con 0x7f81641045e0 2024-04-25T10:54:27.904 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:54:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f81500afbc0 msgr2=0x7f81500b20d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f81500afbc0 0x7f81500b20d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 msgr2=0x7f816419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f816419a250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8160006010 tx=0x7f8160004270 comp rx=0 tx=0).stop 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 shutdown_connections 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81641098a0 0x7f81641a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8164106f40 0x7f816419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f81500afbc0 0x7f81500b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 --2- 172.21.15.19:0/1786393262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81641045e0 0x7f816419a250 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 >> 172.21.15.19:0/1786393262 conn(0x7f81640fdbc0 msgr2=0x7f81641052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 shutdown_connections 2024-04-25T10:54:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:27.906+0000 7f816b355640 1 -- 172.21.15.19:0/1786393262 wait complete. 2024-04-25T10:54:27.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:54:28.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 -- 172.21.15.19:0/594125083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb28103c30 msgr2=0x7fcb28104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/594125083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb28103c30 0x7fcb28104040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009900 tx=0x7fcb1002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 -- 172.21.15.19:0/594125083 shutdown_connections 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/594125083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb28108930 0x7fcb2810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/594125083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb28103c30 0x7fcb28104040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/594125083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb281029f0 0x7fcb281036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 -- 172.21.15.19:0/594125083 >> 172.21.15.19:0/594125083 conn(0x7fcb280fdc00 msgr2=0x7fcb28100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 -- 172.21.15.19:0/594125083 shutdown_connections 2024-04-25T10:54:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 -- 172.21.15.19:0/594125083 wait complete. 2024-04-25T10:54:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.033+0000 7fcb2ce20640 1 Processor -- start 2024-04-25T10:54:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2ce20640 1 -- start start 2024-04-25T10:54:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2ce20640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb281029f0 0x7fcb280ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2ce20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb28103c30 0x7fcb280fe130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2ce20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28108930 0x7fcb280fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2ce20640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2810e4f0 con 0x7fcb281029f0 2024-04-25T10:54:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2ce20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2810e370 con 0x7fcb28103c30 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2ce20640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2810e670 con 0x7fcb28108930 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb281029f0 0x7fcb280ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb281029f0 0x7fcb280ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43250/0 (socket says 172.21.15.19:43250) 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 -- 172.21.15.19:0/3071515853 learned_addr learned my addr 172.21.15.19:0/3071515853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb1ffff640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb28103c30 0x7fcb280fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb26d9d640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28108930 0x7fcb280fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 -- 172.21.15.19:0/3071515853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28108930 msgr2=0x7fcb280fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28108930 0x7fcb280fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 -- 172.21.15.19:0/3071515853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb28103c30 msgr2=0x7fcb280fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb28103c30 0x7fcb280fe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 -- 172.21.15.19:0/3071515853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb280feda0 con 0x7fcb281029f0 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.034+0000 7fcb2659c640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb281029f0 0x7fcb280ffa90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.035+0000 7fcb1f7fe640 1 -- 172.21.15.19:0/3071515853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800ccc0 con 0x7fcb281029f0 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.035+0000 7fcb1f7fe640 1 -- 172.21.15.19:0/3071515853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb180035f0 con 0x7fcb281029f0 2024-04-25T10:54:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.035+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb281a42d0 con 0x7fcb281029f0 2024-04-25T10:54:28.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.035+0000 7fcb1f7fe640 1 -- 172.21.15.19:0/3071515853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800ce60 con 0x7fcb281029f0 2024-04-25T10:54:28.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.035+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb281a4790 con 0x7fcb281029f0 2024-04-25T10:54:28.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.036+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb281a4ab0 con 0x7fcb281029f0 2024-04-25T10:54:28.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.037+0000 7fcb1f7fe640 1 -- 172.21.15.19:0/3071515853 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7fcb18003aa0 con 0x7fcb281029f0 2024-04-25T10:54:28.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.038+0000 7fcb1f7fe640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb140afb40 0x7fcb140b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.038+0000 7fcb1ffff640 1 -- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb140afb40 msgr2=0x7fcb140b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:28.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.038+0000 7fcb1f7fe640 1 -- 172.21.15.19:0/3071515853 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fcb180df1a0 con 0x7fcb281029f0 2024-04-25T10:54:28.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.038+0000 7fcb1ffff640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb140afb40 0x7fcb140b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.041+0000 7fcb1f7fe640 1 -- 172.21.15.19:0/3071515853 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb18053ff0 con 0x7fcb281029f0 2024-04-25T10:54:28.058 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:27.713+0000 7f1c8cc22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:54:28.059 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:27.848+0000 7f1c8cc22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:54:28.059 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:27.945+0000 7f1c8cc22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:54:28.155 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:27.805+0000 7f85b9a22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:54:28.156 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:27.939+0000 7f85b9a22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:54:28.156 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:28.036+0000 7f85b9a22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:54:28.156 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:28.155+0000 7f85b9a22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:54:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.227+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcb281a4c80 con 0x7fcb281029f0 2024-04-25T10:54:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.228+0000 7fcb1f7fe640 1 -- 172.21.15.19:0/3071515853 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fcb180a1e20 con 0x7fcb281029f0 2024-04-25T10:54:28.229 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb140afb40 msgr2=0x7fcb140b2050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb140afb40 0x7fcb140b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb281029f0 msgr2=0x7fcb280ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb281029f0 0x7fcb280ffa90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 shutdown_connections 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28108930 0x7fcb280fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb28103c30 0x7fcb280fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7fcb140afb40 0x7fcb140b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 --2- 172.21.15.19:0/3071515853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb281029f0 0x7fcb280ffa90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 >> 172.21.15.19:0/3071515853 conn(0x7fcb280fdc00 msgr2=0x7fcb28109ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 shutdown_connections 2024-04-25T10:54:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.230+0000 7fcb2ce20640 1 -- 172.21.15.19:0/3071515853 wait complete. 2024-04-25T10:54:28.241 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:27.883+0000 7f48be647200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:54:28.241 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:27.987+0000 7f48be647200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:54:28.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:54:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 -- 172.21.15.19:0/2215887230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638104d20 msgr2=0x7f9638105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 --2- 172.21.15.19:0/2215887230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638104d20 0x7f9638105130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9628009900 tx=0x7f962802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 -- 172.21.15.19:0/2215887230 shutdown_connections 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 --2- 172.21.15.19:0/2215887230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9638109a20 0x7f963810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 --2- 172.21.15.19:0/2215887230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638104d20 0x7f9638105130 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 --2- 172.21.15.19:0/2215887230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9638103ae0 0x7f96381047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 -- 172.21.15.19:0/2215887230 >> 172.21.15.19:0/2215887230 conn(0x7f96380fecd0 msgr2=0x7f9638101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 -- 172.21.15.19:0/2215887230 shutdown_connections 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.358+0000 7f963fb26640 1 -- 172.21.15.19:0/2215887230 wait complete. 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 Processor -- start 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 -- start start 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638103ae0 0x7f963819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9638104d20 0x7f963819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9638109a20 0x7f96381a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f963810f610 con 0x7f9638103ae0 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f963810f490 con 0x7f9638104d20 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963fb26640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f963810f790 con 0x7f9638109a20 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963d8c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638103ae0 0x7f963819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963d8c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638103ae0 0x7f963819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43276/0 (socket says 172.21.15.19:43276) 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.359+0000 7f963d8c2640 1 -- 172.21.15.19:0/1848551904 learned_addr learned my addr 172.21.15.19:0/1848551904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d0c1640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9638104d20 0x7f963819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963e0c3640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9638109a20 0x7f96381a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d8c2640 1 -- 172.21.15.19:0/1848551904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9638109a20 msgr2=0x7f96381a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d8c2640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9638109a20 0x7f96381a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d8c2640 1 -- 172.21.15.19:0/1848551904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9638104d20 msgr2=0x7f963819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d8c2640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9638104d20 0x7f963819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d8c2640 1 -- 172.21.15.19:0/1848551904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96381a1d30 con 0x7f9638103ae0 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d0c1640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9638104d20 0x7f963819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963d8c2640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638103ae0 0x7f963819a1f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f963400e9f0 tx=0x7f963400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f962effd640 1 -- 172.21.15.19:0/1848551904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963400ccc0 con 0x7f9638103ae0 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96381a1fc0 con 0x7f9638103ae0 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f962effd640 1 -- 172.21.15.19:0/1848551904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96340035f0 con 0x7f9638103ae0 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f962effd640 1 -- 172.21.15.19:0/1848551904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963400ce60 con 0x7f9638103ae0 2024-04-25T10:54:28.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.360+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96381a2480 con 0x7f9638103ae0 2024-04-25T10:54:28.365 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:28.063+0000 7f1c8cc22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:54:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.361+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9638108830 con 0x7f9638103ae0 2024-04-25T10:54:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.363+0000 7f962effd640 1 -- 172.21.15.19:0/1848551904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f9634003aa0 con 0x7f9638103ae0 2024-04-25T10:54:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.364+0000 7f962effd640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 0x7f96240b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.364+0000 7f963d0c1640 1 -- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 msgr2=0x7f96240b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.364+0000 7f962effd640 1 -- 172.21.15.19:0/1848551904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f96340df080 con 0x7f9638103ae0 2024-04-25T10:54:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.364+0000 7f963d0c1640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 0x7f96240b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.372+0000 7f962effd640 1 -- 172.21.15.19:0/1848551904 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96340a7c10 con 0x7f9638103ae0 2024-04-25T10:54:28.520 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:28.258+0000 7f48be647200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:54:28.520 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:28.413+0000 7f48be647200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:54:28.520 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:28.519+0000 7f48be647200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:54:28.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.562+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96380637a0 con 0x7f9638103ae0 2024-04-25T10:54:28.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.563+0000 7f962effd640 1 -- 172.21.15.19:0/1848551904 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f96340a7510 con 0x7f9638103ae0 2024-04-25T10:54:28.563 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:54:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963d0c1640 1 -- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 msgr2=0x7f96240b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963d0c1640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 0x7f96240b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:54:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 msgr2=0x7f96240b2050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 0x7f96240b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638103ae0 msgr2=0x7f963819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638103ae0 0x7f963819a1f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f963400e9f0 tx=0x7f963400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 shutdown_connections 2024-04-25T10:54:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9638109a20 0x7f96381a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9638104d20 0x7f963819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f96240afb40 0x7f96240b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 --2- 172.21.15.19:0/1848551904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9638103ae0 0x7f963819a1f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.565+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 >> 172.21.15.19:0/1848551904 conn(0x7f96380fecd0 msgr2=0x7f9638105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.566+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 shutdown_connections 2024-04-25T10:54:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.566+0000 7f963fb26640 1 -- 172.21.15.19:0/1848551904 wait complete. 2024-04-25T10:54:28.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:54:28.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 -- 172.21.15.19:0/1559698665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c1029c0 msgr2=0x7f1b1c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 --2- 172.21.15.19:0/1559698665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c1029c0 0x7f1b1c1036c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009900 tx=0x7f1b0c02a410 comp rx=0 tx=0).stop 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 -- 172.21.15.19:0/1559698665 shutdown_connections 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 --2- 172.21.15.19:0/1559698665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b1c108900 0x7f1b1c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 --2- 172.21.15.19:0/1559698665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b1c103c00 0x7f1b1c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 --2- 172.21.15.19:0/1559698665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c1029c0 0x7f1b1c1036c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 -- 172.21.15.19:0/1559698665 >> 172.21.15.19:0/1559698665 conn(0x7f1b1c0fdbd0 msgr2=0x7f1b1c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 -- 172.21.15.19:0/1559698665 shutdown_connections 2024-04-25T10:54:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 -- 172.21.15.19:0/1559698665 wait complete. 2024-04-25T10:54:28.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 Processor -- start 2024-04-25T10:54:28.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.691+0000 7f1b24434640 1 -- start start 2024-04-25T10:54:28.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b24434640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b1c1029c0 0x7f1b1c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b24434640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b1c103c00 0x7f1b1c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b24434640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c108900 0x7f1b1c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b24434640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c10e510 con 0x7f1b1c108900 2024-04-25T10:54:28.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b24434640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c10e390 con 0x7f1b1c1029c0 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b24434640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c10e690 con 0x7f1b1c103c00 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c108900 0x7f1b1c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c108900 0x7f1b1c1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43292/0 (socket says 172.21.15.19:43292) 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 -- 172.21.15.19:0/4258094264 learned_addr learned my addr 172.21.15.19:0/4258094264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b221d0640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b1c1029c0 0x7f1b1c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 -- 172.21.15.19:0/4258094264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b1c103c00 msgr2=0x7f1b1c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b1c103c00 0x7f1b1c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 -- 172.21.15.19:0/4258094264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b1c1029c0 msgr2=0x7f1b1c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b1c1029c0 0x7f1b1c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 -- 172.21.15.19:0/4258094264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c1a1d50 con 0x7f1b1c108900 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.692+0000 7f1b229d1640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c108900 0x7f1b1c1a16e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400eb10 tx=0x7f1b1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.693+0000 7f1b137fe640 1 -- 172.21.15.19:0/4258094264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14004b20 con 0x7f1b1c108900 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.693+0000 7f1b137fe640 1 -- 172.21.15.19:0/4258094264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b14004cc0 con 0x7f1b1c108900 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.693+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c1a1fe0 con 0x7f1b1c108900 2024-04-25T10:54:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.693+0000 7f1b137fe640 1 -- 172.21.15.19:0/4258094264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14010670 con 0x7f1b1c108900 2024-04-25T10:54:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.693+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1a2500 con 0x7f1b1c108900 2024-04-25T10:54:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.694+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c1a2790 con 0x7f1b1c108900 2024-04-25T10:54:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.695+0000 7f1b137fe640 1 -- 172.21.15.19:0/4258094264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f1b14005000 con 0x7f1b1c108900 2024-04-25T10:54:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.696+0000 7f1b137fe640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1b080afbc0 0x7f1b080b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.696+0000 7f1b221d0640 1 -- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1b080afbc0 msgr2=0x7f1b080b20d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.696+0000 7f1b137fe640 1 -- 172.21.15.19:0/4258094264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f1b140df410 con 0x7f1b1c108900 2024-04-25T10:54:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.696+0000 7f1b221d0640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1b080afbc0 0x7f1b080b20d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:28.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.699+0000 7f1b137fe640 1 -- 172.21.15.19:0/4258094264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b14053ff0 con 0x7f1b1c108900 2024-04-25T10:54:28.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:28.464+0000 7f85b9a22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:54:28.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:28 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1786393262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:28 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3071515853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:28 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1848551904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:28 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1786393262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:28 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3071515853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:28 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1848551904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.809 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:28.362+0000 7f1c8cc22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:54:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1786393262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3071515853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1848551904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:28.861 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:28.648+0000 7f48be647200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:54:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.887+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b1c1a2960 con 0x7f1b1c108900 2024-04-25T10:54:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.889+0000 7f1b137fe640 1 -- 172.21.15.19:0/4258094264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22659 (secure 0 0 0) 0x7f1b140a7fa0 con 0x7f1b1c108900 2024-04-25T10:54:28.890 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:28.890 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:53:31.142322+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1b080afbc0 msgr2=0x7f1b080b20d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1b080afbc0 0x7f1b080b20d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c108900 msgr2=0x7f1b1c1a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c108900 0x7f1b1c1a16e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400eb10 tx=0x7f1b1400c2d0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 shutdown_connections 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f1b080afbc0 0x7f1b080b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b1c108900 0x7f1b1c1a16e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b1c103c00 0x7f1b1c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 --2- 172.21.15.19:0/4258094264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b1c1029c0 0x7f1b1c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 >> 172.21.15.19:0/4258094264 conn(0x7f1b1c0fdbd0 msgr2=0x7f1b1c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 shutdown_connections 2024-04-25T10:54:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:28.891+0000 7f1b24434640 1 -- 172.21.15.19:0/4258094264 wait complete. 2024-04-25T10:54:28.904 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:54:28.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 -- 172.21.15.19:0/2537662761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 msgr2=0x7f2e18104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 --2- 172.21.15.19:0/2537662761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 0x7f2e18104040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009900 tx=0x7f2e0c02a410 comp rx=0 tx=0).stop 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 -- 172.21.15.19:0/2537662761 shutdown_connections 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 --2- 172.21.15.19:0/2537662761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e18108930 0x7f2e1810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 --2- 172.21.15.19:0/2537662761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 0x7f2e18104040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 --2- 172.21.15.19:0/2537662761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e181029f0 0x7f2e181036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 -- 172.21.15.19:0/2537662761 >> 172.21.15.19:0/2537662761 conn(0x7f2e180fdbe0 msgr2=0x7f2e18100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 -- 172.21.15.19:0/2537662761 shutdown_connections 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.021+0000 7f2e1d364640 1 -- 172.21.15.19:0/2537662761 wait complete. 2024-04-25T10:54:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 Processor -- start 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 -- start start 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181029f0 0x7f2e1819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 0x7f2e1819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e18108930 0x7f2e181a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1810f620 con 0x7f2e18103c30 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1810f4a0 con 0x7f2e181029f0 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e1d364640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1810f7a0 con 0x7f2e18108930 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e167fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 0x7f2e1819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e167fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 0x7f2e1819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43314/0 (socket says 172.21.15.19:43314) 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e16ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181029f0 0x7f2e1819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e167fc640 1 -- 172.21.15.19:0/1795476847 learned_addr learned my addr 172.21.15.19:0/1795476847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.022+0000 7f2e177fe640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e18108930 0x7f2e181a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e16ffd640 1 -- 172.21.15.19:0/1795476847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e18108930 msgr2=0x7f2e181a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e16ffd640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e18108930 0x7f2e181a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e16ffd640 1 -- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 msgr2=0x7f2e1819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e16ffd640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 0x7f2e1819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e16ffd640 1 -- 172.21.15.19:0/1795476847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e181a1de0 con 0x7f2e181029f0 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e16ffd640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181029f0 0x7f2e1819a2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e181022c0 tx=0x7f2e0800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2dfffff640 1 -- 172.21.15.19:0/1795476847 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800cc40 con 0x7f2e181029f0 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2dfffff640 1 -- 172.21.15.19:0/1795476847 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e0800cde0 con 0x7f2e181029f0 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2dfffff640 1 -- 172.21.15.19:0/1795476847 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08010430 con 0x7f2e181029f0 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e181a20d0 con 0x7f2e181029f0 2024-04-25T10:54:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.023+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e181a25d0 con 0x7f2e181029f0 2024-04-25T10:54:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.024+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e18107740 con 0x7f2e181029f0 2024-04-25T10:54:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.026+0000 7f2dfffff640 1 -- 172.21.15.19:0/1795476847 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f2e08003680 con 0x7f2e181029f0 2024-04-25T10:54:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.026+0000 7f2dfffff640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2e000afb40 0x7f2e000b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.026+0000 7f2e167fc640 1 -- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2e000afb40 msgr2=0x7f2e000b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.026+0000 7f2dfffff640 1 -- 172.21.15.19:0/1795476847 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f2e080df0a0 con 0x7f2e181029f0 2024-04-25T10:54:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.026+0000 7f2e167fc640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2e000afb40 0x7f2e000b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:29.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.030+0000 7f2dfffff640 1 -- 172.21.15.19:0/1795476847 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e080a7c90 con 0x7f2e181029f0 2024-04-25T10:54:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.217+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e180637a0 con 0x7f2e181029f0 2024-04-25T10:54:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.218+0000 7f2dfffff640 1 -- 172.21.15.19:0/1795476847 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f2e080a7590 con 0x7f2e181029f0 2024-04-25T10:54:29.219 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:54:29.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2e000afb40 msgr2=0x7f2e000b2050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:29.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2e000afb40 0x7f2e000b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181029f0 msgr2=0x7f2e1819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181029f0 0x7f2e1819a2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e181022c0 tx=0x7f2e0800ee50 comp rx=0 tx=0).stop 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 shutdown_connections 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e18108930 0x7f2e181a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f2e000afb40 0x7f2e000b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e18103c30 0x7f2e1819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 --2- 172.21.15.19:0/1795476847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181029f0 0x7f2e1819a2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 >> 172.21.15.19:0/1795476847 conn(0x7f2e180fdbe0 msgr2=0x7f2e18104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 shutdown_connections 2024-04-25T10:54:29.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.220+0000 7f2e1d364640 1 -- 172.21.15.19:0/1795476847 wait complete. 2024-04-25T10:54:29.231 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:28.972+0000 7f48be647200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:54:29.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:54:29.309 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:28.965+0000 7f1c8cc22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:54:29.309 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y[34180]: 2024-04-25T10:54:29.183+0000 7f1c8cc22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:54:29.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.347+0000 7ff626049640 1 -- 172.21.15.19:0/3454110353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620103c20 msgr2=0x7ff620104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.347+0000 7ff626049640 1 --2- 172.21.15.19:0/3454110353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620103c20 0x7ff620104030 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff608009900 tx=0x7ff60802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:29.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.347+0000 7ff626049640 1 -- 172.21.15.19:0/3454110353 shutdown_connections 2024-04-25T10:54:29.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.347+0000 7ff626049640 1 --2- 172.21.15.19:0/3454110353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff620108920 0x7ff62010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.347+0000 7ff626049640 1 --2- 172.21.15.19:0/3454110353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620103c20 0x7ff620104030 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.347+0000 7ff626049640 1 --2- 172.21.15.19:0/3454110353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6201029e0 0x7ff6201036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.347+0000 7ff626049640 1 -- 172.21.15.19:0/3454110353 >> 172.21.15.19:0/3454110353 conn(0x7ff6200fdbd0 msgr2=0x7ff620100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:29.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.348+0000 7ff626049640 1 -- 172.21.15.19:0/3454110353 shutdown_connections 2024-04-25T10:54:29.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.348+0000 7ff626049640 1 -- 172.21.15.19:0/3454110353 wait complete. 2024-04-25T10:54:29.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.348+0000 7ff626049640 1 Processor -- start 2024-04-25T10:54:29.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.348+0000 7ff626049640 1 -- start start 2024-04-25T10:54:29.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.348+0000 7ff626049640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6201029e0 0x7ff62006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff626049640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff620103c20 0x7ff62019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff626049640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620108920 0x7ff6201a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff626049640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010e4e0 con 0x7ff620108920 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff626049640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010e360 con 0x7ff620103c20 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff626049640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010e660 con 0x7ff6201029e0 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620108920 0x7ff6201a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620108920 0x7ff6201a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43332/0 (socket says 172.21.15.19:43332) 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 -- 172.21.15.19:0/1282671476 learned_addr learned my addr 172.21.15.19:0/1282671476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61f7fe640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6201029e0 0x7ff62006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff616ffd640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff620103c20 0x7ff62019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 -- 172.21.15.19:0/1282671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6201029e0 msgr2=0x7ff62006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6201029e0 0x7ff62006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 -- 172.21.15.19:0/1282671476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff620103c20 msgr2=0x7ff62019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff620103c20 0x7ff62019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 -- 172.21.15.19:0/1282671476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6201a2070 con 0x7ff620108920 2024-04-25T10:54:29.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61ffff640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620108920 0x7ff6201a18a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff61800eb10 tx=0x7ff61800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:29.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61d7fa640 1 -- 172.21.15.19:0/1282671476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618004b20 con 0x7ff620108920 2024-04-25T10:54:29.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6201a2360 con 0x7ff620108920 2024-04-25T10:54:29.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61d7fa640 1 -- 172.21.15.19:0/1282671476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff618004cc0 con 0x7ff620108920 2024-04-25T10:54:29.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.349+0000 7ff61d7fa640 1 -- 172.21.15.19:0/1282671476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618010670 con 0x7ff620108920 2024-04-25T10:54:29.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.350+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6201a28b0 con 0x7ff620108920 2024-04-25T10:54:29.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.350+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff620107730 con 0x7ff620108920 2024-04-25T10:54:29.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.352+0000 7ff61d7fa640 1 -- 172.21.15.19:0/1282671476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7ff618005000 con 0x7ff620108920 2024-04-25T10:54:29.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.353+0000 7ff61d7fa640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff60c0afb40 0x7ff60c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.353+0000 7ff61f7fe640 1 -- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff60c0afb40 msgr2=0x7ff60c0b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:29.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.353+0000 7ff61d7fa640 1 -- 172.21.15.19:0/1282671476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7ff6180df4c0 con 0x7ff620108920 2024-04-25T10:54:29.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.353+0000 7ff61f7fe640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff60c0afb40 0x7ff60c0b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.356+0000 7ff61d7fa640 1 -- 172.21.15.19:0/1282671476 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff618053ff0 con 0x7ff620108920 2024-04-25T10:54:29.501 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:29.071+0000 7f85b9a22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:54:29.501 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z[34905]: 2024-04-25T10:54:29.291+0000 7f85b9a22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:54:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.545+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff62002dcd0 con 0x7ff620108920 2024-04-25T10:54:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.546+0000 7ff61d7fa640 1 -- 172.21.15.19:0/1282671476 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7ff6180a80b0 con 0x7ff620108920 2024-04-25T10:54:29.546 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff60c0afb40 msgr2=0x7ff60c0b2050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff60c0afb40 0x7ff60c0b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620108920 msgr2=0x7ff6201a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620108920 0x7ff6201a18a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff61800eb10 tx=0x7ff61800c2d0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 shutdown_connections 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7ff60c0afb40 0x7ff60c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff620108920 0x7ff6201a18a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff620103c20 0x7ff62019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 --2- 172.21.15.19:0/1282671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6201029e0 0x7ff62006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 >> 172.21.15.19:0/1282671476 conn(0x7ff6200fdbd0 msgr2=0x7ff620104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 shutdown_connections 2024-04-25T10:54:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.548+0000 7ff626049640 1 -- 172.21.15.19:0/1282671476 wait complete. 2024-04-25T10:54:29.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 -- 172.21.15.19:0/2649093933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c102350 msgr2=0x7f149c1090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 --2- 172.21.15.19:0/2649093933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c102350 0x7f149c1090e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1488009900 tx=0x7f148802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 -- 172.21.15.19:0/2649093933 shutdown_connections 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 --2- 172.21.15.19:0/2649093933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c109770 0x7f149c10bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 --2- 172.21.15.19:0/2649093933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c102350 0x7f149c1090e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 --2- 172.21.15.19:0/2649093933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c101e50 0x7f149c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 -- 172.21.15.19:0/2649093933 >> 172.21.15.19:0/2649093933 conn(0x7f149c0fd340 msgr2=0x7f149c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 -- 172.21.15.19:0/2649093933 shutdown_connections 2024-04-25T10:54:29.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.680+0000 7f14a2c49640 1 -- 172.21.15.19:0/2649093933 wait complete. 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 Processor -- start 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 -- start start 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c101e50 0x7f149c104320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c102350 0x7f149c1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c109770 0x7f149c102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f149c10f5c0 con 0x7f149c101e50 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f149c10f440 con 0x7f149c109770 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a2c49640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f149c10f740 con 0x7f149c102350 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c101e50 0x7f149c104320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c101e50 0x7f149c104320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43352/0 (socket says 172.21.15.19:43352) 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 -- 172.21.15.19:0/458100236 learned_addr learned my addr 172.21.15.19:0/458100236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a11e6640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c109770 0x7f149c102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f149bfff640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c102350 0x7f149c1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 -- 172.21.15.19:0/458100236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c102350 msgr2=0x7f149c1029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c102350 0x7f149c1029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 -- 172.21.15.19:0/458100236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c109770 msgr2=0x7f149c102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c109770 0x7f149c102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.681+0000 7f14a09e5640 1 -- 172.21.15.19:0/458100236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c103630 con 0x7f149c101e50 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.682+0000 7f14a09e5640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c101e50 0x7f149c104320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f149000e9f0 tx=0x7f149000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.682+0000 7f1499ffb640 1 -- 172.21.15.19:0/458100236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149000ccc0 con 0x7f149c101e50 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.682+0000 7f1499ffb640 1 -- 172.21.15.19:0/458100236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14900108e0 con 0x7f149c101e50 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.682+0000 7f1499ffb640 1 -- 172.21.15.19:0/458100236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490018880 con 0x7f149c101e50 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.682+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149c1a42b0 con 0x7f149c101e50 2024-04-25T10:54:29.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.682+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149c1a4770 con 0x7f149c101e50 2024-04-25T10:54:29.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.683+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149c0fee60 con 0x7f149c101e50 2024-04-25T10:54:29.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.685+0000 7f1499ffb640 1 -- 172.21.15.19:0/458100236 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 147537+0+0 (secure 0 0 0) 0x7f1490010430 con 0x7f149c101e50 2024-04-25T10:54:29.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.686+0000 7f1499ffb640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 0x7f148c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:29.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.686+0000 7f149bfff640 1 -- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 msgr2=0x7f148c0b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:29.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.686+0000 7f1499ffb640 1 -- 172.21.15.19:0/458100236 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f14900df520 con 0x7f149c101e50 2024-04-25T10:54:29.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.686+0000 7f149bfff640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 0x7f148c0b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:54:29.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.689+0000 7f1499ffb640 1 -- 172.21.15.19:0/458100236 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14900a80f0 con 0x7f149c101e50 2024-04-25T10:54:29.861 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:29.638+0000 7f48be647200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:54:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4258094264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: Standby manager daemon y restarted 2024-04-25T10:54:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: Standby manager daemon y started 2024-04-25T10:54:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:54:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:54:29.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1795476847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: Standby manager daemon z restarted 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: Standby manager daemon z started 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:54:29.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1282671476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.887+0000 7f149bfff640 1 -- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 msgr2=0x7f148c0b2050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1708221429 2024-04-25T10:54:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.887+0000 7f149bfff640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 0x7f148c0b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:54:29.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.899+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f149c02dce0 con 0x7f149c101e50 2024-04-25T10:54:29.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.899+0000 7f1499ffb640 1 -- 172.21.15.19:0/458100236 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f149c02dce0 con 0x7f149c101e50 2024-04-25T10:54:29.900 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:54:29.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 msgr2=0x7f148c0b2050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:54:29.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 0x7f148c0b2050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c101e50 msgr2=0x7f149c104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:29.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c101e50 0x7f149c104320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f149000e9f0 tx=0x7f149000edf0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 shutdown_connections 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f149c109770 0x7f149c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f149c102350 0x7f149c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:6800/1708221429,v1:172.21.15.19:6801/1708221429] conn(0x7f148c0afb40 0x7f148c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 --2- 172.21.15.19:0/458100236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c101e50 0x7f149c104320 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.902+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 >> 172.21.15.19:0/458100236 conn(0x7f149c0fd340 msgr2=0x7f149c10a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.903+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 shutdown_connections 2024-04-25T10:54:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:29.903+0000 7f14a2c49640 1 -- 172.21.15.19:0/458100236 wait complete. 2024-04-25T10:54:29.919 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-25T10:54:29.920 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:54:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4258094264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: Standby manager daemon y restarted 2024-04-25T10:54:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: Standby manager daemon y started 2024-04-25T10:54:30.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1795476847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: Standby manager daemon z restarted 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: Standby manager daemon z started 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:54:30.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:29 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1282671476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.048+0000 7f12b0caa640 1 -- 172.21.15.19:0/790482888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac103c00 msgr2=0x7f12ac104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.048+0000 7f12b0caa640 1 --2- 172.21.15.19:0/790482888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac103c00 0x7f12ac104010 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009900 tx=0x7f12a002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.048+0000 7f12b0caa640 1 -- 172.21.15.19:0/790482888 shutdown_connections 2024-04-25T10:54:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.049+0000 7f12b0caa640 1 --2- 172.21.15.19:0/790482888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac108900 0x7f12ac10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.049+0000 7f12b0caa640 1 --2- 172.21.15.19:0/790482888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac103c00 0x7f12ac104010 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.049+0000 7f12b0caa640 1 --2- 172.21.15.19:0/790482888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12ac1029c0 0x7f12ac1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.049+0000 7f12b0caa640 1 -- 172.21.15.19:0/790482888 >> 172.21.15.19:0/790482888 conn(0x7f12ac0fdbd0 msgr2=0x7f12ac100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.049+0000 7f12b0caa640 1 -- 172.21.15.19:0/790482888 shutdown_connections 2024-04-25T10:54:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.049+0000 7f12b0caa640 1 -- 172.21.15.19:0/790482888 wait complete. 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 Processor -- start 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 -- start start 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac1029c0 0x7f12ac19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac103c00 0x7f12ac19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12ac108900 0x7f12ac1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac10f650 con 0x7f12ac1029c0 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac10f4d0 con 0x7f12ac108900 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12b0caa640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac10f7d0 con 0x7f12ac103c00 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.050+0000 7f12aa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac1029c0 0x7f12ac19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac103c00 0x7f12ac19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac103c00 0x7f12ac19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:42002/0 (socket says 172.21.15.19:42002) 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 -- 172.21.15.19:0/1596156808 learned_addr learned my addr 172.21.15.19:0/1596156808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12aad9d640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12ac108900 0x7f12ac1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 -- 172.21.15.19:0/1596156808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12ac108900 msgr2=0x7f12ac1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12ac108900 0x7f12ac1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 -- 172.21.15.19:0/1596156808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac1029c0 msgr2=0x7f12ac19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac1029c0 0x7f12ac19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 -- 172.21.15.19:0/1596156808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac1a1d30 con 0x7f12ac103c00 2024-04-25T10:54:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.051+0000 7f12a9d9b640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac103c00 0x7f12ac19a730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f12a0031440 tx=0x7f12a0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.052+0000 7f129b7fe640 1 -- 172.21.15.19:0/1596156808 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a003f070 con 0x7f12ac103c00 2024-04-25T10:54:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.052+0000 7f129b7fe640 1 -- 172.21.15.19:0/1596156808 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a0031c50 con 0x7f12ac103c00 2024-04-25T10:54:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.052+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac1a1fc0 con 0x7f12ac103c00 2024-04-25T10:54:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.052+0000 7f129b7fe640 1 -- 172.21.15.19:0/1596156808 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0035030 con 0x7f12ac103c00 2024-04-25T10:54:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.052+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac1a23d0 con 0x7f12ac103c00 2024-04-25T10:54:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.053+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac107710 con 0x7f12ac103c00 2024-04-25T10:54:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.054+0000 7f129b7fe640 1 -- 172.21.15.19:0/1596156808 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 21) ==== 147205+0+0 (secure 0 0 0) 0x7f12a0042070 con 0x7f12ac103c00 2024-04-25T10:54:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.055+0000 7f129b7fe640 1 -- 172.21.15.19:0/1596156808 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f12a00ff010 con 0x7f12ac103c00 2024-04-25T10:54:30.058 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4258094264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: Standby manager daemon y restarted 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: Standby manager daemon y started 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.112:0/3367246531' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1795476847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: Standby manager daemon z restarted 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: Standby manager daemon z started 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='mgr.? 172.21.15.169:0/2406114251' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:54:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:29 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1282671476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.061+0000 7f129b7fe640 1 -- 172.21.15.19:0/1596156808 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12a00c78b0 con 0x7f12ac103c00 2024-04-25T10:54:30.196 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:29.899+0000 7f48be647200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:54:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.337+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12ac02db50 con 0x7f12ac103c00 2024-04-25T10:54:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.338+0000 7f129b7fe640 1 -- 172.21.15.19:0/1596156808 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f12a00cbbb0 con 0x7f12ac103c00 2024-04-25T10:54:30.338 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:30.338 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:54:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac103c00 msgr2=0x7f12ac19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac103c00 0x7f12ac19a730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f12a0031440 tx=0x7f12a0003c10 comp rx=0 tx=0).stop 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 shutdown_connections 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12ac108900 0x7f12ac1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac103c00 0x7f12ac19a730 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 --2- 172.21.15.19:0/1596156808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12ac1029c0 0x7f12ac19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 >> 172.21.15.19:0/1596156808 conn(0x7f12ac0fdbd0 msgr2=0x7f12ac1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 shutdown_connections 2024-04-25T10:54:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.340+0000 7f12b0caa640 1 -- 172.21.15.19:0/1596156808 wait complete. 2024-04-25T10:54:30.355 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-25T10:54:30.355 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-25T10:54:30.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:54:30.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 -- 172.21.15.19:0/4101843409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8109a10 msgr2=0x7f19f810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 --2- 172.21.15.19:0/4101843409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8109a10 0x7f19f810be30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009ca0 tx=0x7f19f002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 -- 172.21.15.19:0/4101843409 shutdown_connections 2024-04-25T10:54:30.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 --2- 172.21.15.19:0/4101843409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8109a10 0x7f19f810be30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 --2- 172.21.15.19:0/4101843409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8104d10 0x7f19f8105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 --2- 172.21.15.19:0/4101843409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8103ad0 0x7f19f81047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 -- 172.21.15.19:0/4101843409 >> 172.21.15.19:0/4101843409 conn(0x7f19f80fecc0 msgr2=0x7f19f8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:30.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 -- 172.21.15.19:0/4101843409 shutdown_connections 2024-04-25T10:54:30.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.470+0000 7f1a00364640 1 -- 172.21.15.19:0/4101843409 wait complete. 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 Processor -- start 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 -- start start 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8103ad0 0x7f19f819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8104d10 0x7f19f819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8109a10 0x7f19f81a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810f630 con 0x7f19f8103ad0 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810f4b0 con 0x7f19f8104d10 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f1a00364640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810f7b0 con 0x7f19f8109a10 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f19fe100640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8103ad0 0x7f19f819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:30.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.471+0000 7f19fe100640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8103ad0 0x7f19f819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43502/0 (socket says 172.21.15.19:43502) 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe100640 1 -- 172.21.15.19:0/2924402869 learned_addr learned my addr 172.21.15.19:0/2924402869 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe901640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8109a10 0x7f19f81a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fd8ff640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8104d10 0x7f19f819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe100640 1 -- 172.21.15.19:0/2924402869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8109a10 msgr2=0x7f19f81a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe100640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8109a10 0x7f19f81a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe100640 1 -- 172.21.15.19:0/2924402869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8104d10 msgr2=0x7f19f819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe100640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8104d10 0x7f19f819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe100640 1 -- 172.21.15.19:0/2924402869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f81a1ce0 con 0x7f19f8103ad0 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fd8ff640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8104d10 0x7f19f819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19fe100640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8103ad0 0x7f19f819a1a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005fd0 tx=0x7f19f400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4017070 con 0x7f19f8103ad0 2024-04-25T10:54:30.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f4009c30 con 0x7f19f8103ad0 2024-04-25T10:54:30.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f81a1fd0 con 0x7f19f8103ad0 2024-04-25T10:54:30.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4003d10 con 0x7f19f8103ad0 2024-04-25T10:54:30.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.472+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f81a2430 con 0x7f19f8103ad0 2024-04-25T10:54:30.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.473+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f8069310 con 0x7f19f8103ad0 2024-04-25T10:54:30.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.475+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 21) ==== 147205+0+0 (secure 0 0 0) 0x7f19f4003eb0 con 0x7f19f8103ad0 2024-04-25T10:54:30.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.476+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f19f40deae0 con 0x7f19f8103ad0 2024-04-25T10:54:30.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.484+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19f4020030 con 0x7f19f8103ad0 2024-04-25T10:54:30.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f4820554700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f4820554700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f4820554700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f4820554700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f4820554700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f481d50e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f481d50e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.602 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f481d50e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.602 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f481d50e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.602 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:54:30.206+0000 7f481d50e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:54:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-25T10:54:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/458100236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: Active manager daemon x restarted 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: Activating manager daemon x 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: mgrmap e21: x(active, starting, since 0.0165489s), standbys: z, y 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:54:30.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:54:30.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: Manager daemon x is now available 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:54:30.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:54:30.865 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:30 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1596156808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:30.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.919+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f19f40a2c40 con 0x7f19f8103ad0 2024-04-25T10:54:30.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.920+0000 7f19ef7fe640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19e40b6830 0x7f19e40b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:30.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.920+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19f4011ec0 con 0x7f19e40b6830 2024-04-25T10:54:30.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.927+0000 7f19fd8ff640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19e40b6830 0x7f19e40b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:30.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.931+0000 7f19fd8ff640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19e40b6830 0x7f19e40b8c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f19f819b240 tx=0x7f19e8005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:30.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.945+0000 7f19ef7fe640 1 -- 172.21.15.19:0/2924402869 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 Warning: due to ceph-mgr restart, some PG states may not be up to date 2024-04-25T10:54:30.946 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all) ==== 89+0+230503 (secure 0 0 0) 0x7f19f4011ec0 con 0x7f19e40b6830 2024-04-25T10:54:30.949 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:Warning: due to ceph-mgr restart, some PG states may not be up to date 2024-04-25T10:54:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:54:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19e40b6830 msgr2=0x7f19e40b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19e40b6830 0x7f19e40b8c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f19f819b240 tx=0x7f19e8005f90 comp rx=0 tx=0).stop 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8103ad0 msgr2=0x7f19f819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8103ad0 0x7f19f819a1a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005fd0 tx=0x7f19f400dde0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 shutdown_connections 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19e40b6830 0x7f19e40b8c50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8109a10 0x7f19f81a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8104d10 0x7f19f819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 --2- 172.21.15.19:0/2924402869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f8103ad0 0x7f19f819a1a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 >> 172.21.15.19:0/2924402869 conn(0x7f19f80fecc0 msgr2=0x7f19f81059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 shutdown_connections 2024-04-25T10:54:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:30.949+0000 7f1a00364640 1 -- 172.21.15.19:0/2924402869 wait complete. 2024-04-25T10:54:30.961 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":false,"pg_map":{"version":2,"stamp":"2024-04-25T10:54:29.930552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:54:29.901359+0000","last_change":"2024-04-25T10:54:29.901359+0000","last_active":"2024-04-25T10:54:29.901359+0000","last_peered":"2024-04-25T10:54:29.901359+0000","last_clean":"2024-04-25T10:54:29.901359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:54:29.901359+0000","last_undegraded":"2024-04-25T10:54:29.901359+0000","last_fullsized":"2024-04-25T10:54:29.901359+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:54:29.901359+0000","last_clean_scrub_stamp":"2024-04-25T10:54:29.901359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2024-04-25T10:54:30.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:54:31.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/458100236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: Active manager daemon x restarted 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: Activating manager daemon x 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: mgrmap e21: x(active, starting, since 0.0165489s), standbys: z, y 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:54:31.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:54:31.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: Manager daemon x is now available 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:54:31.004 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:54:31.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:30 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1596156808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:31.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/458100236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: Active manager daemon x restarted 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: Activating manager daemon x 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: mgrmap e21: x(active, starting, since 0.0165489s), standbys: z, y 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:54:31.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:54:31.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:54:31.062 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: Manager daemon x is now available 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:54:31.063 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:30 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1596156808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:31.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 -- 172.21.15.19:0/2762062566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 msgr2=0x7f6b2c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/2762062566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c1036c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009900 tx=0x7f6b1402a410 comp rx=0 tx=0).stop 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 -- 172.21.15.19:0/2762062566 shutdown_connections 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/2762062566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b2c108900 0x7f6b2c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/2762062566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c103c00 0x7f6b2c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/2762062566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c1036c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 -- 172.21.15.19:0/2762062566 >> 172.21.15.19:0/2762062566 conn(0x7f6b2c0fdbd0 msgr2=0x7f6b2c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 -- 172.21.15.19:0/2762062566 shutdown_connections 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 -- 172.21.15.19:0/2762062566 wait complete. 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 Processor -- start 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.125+0000 7f6b32c6e640 1 -- start start 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b32c6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b32c6e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b2c103c00 0x7f6b2c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b32c6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c108900 0x7f6b2c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b32c6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10e510 con 0x7f6b2c1029c0 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b32c6e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10e390 con 0x7f6b2c108900 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b32c6e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10e690 con 0x7f6b2c103c00 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43506/0 (socket says 172.21.15.19:43506) 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 -- 172.21.15.19:0/3278768839 learned_addr learned my addr 172.21.15.19:0/3278768839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b2bfff640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b2c103c00 0x7f6b2c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b3120b640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c108900 0x7f6b2c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 -- 172.21.15.19:0/3278768839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b2c103c00 msgr2=0x7f6b2c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b2c103c00 0x7f6b2c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 -- 172.21.15.19:0/3278768839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c108900 msgr2=0x7f6b2c1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c108900 0x7f6b2c1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 -- 172.21.15.19:0/3278768839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c1a0c40 con 0x7f6b2c1029c0 2024-04-25T10:54:31.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b3120b640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c108900 0x7f6b2c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:31.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b30a0a640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c199150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6b14031440 tx=0x7f6b14003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:31.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b29ffb640 1 -- 172.21.15.19:0/3278768839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1403f070 con 0x7f6b2c1029c0 2024-04-25T10:54:31.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b29ffb640 1 -- 172.21.15.19:0/3278768839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b14031c50 con 0x7f6b2c1029c0 2024-04-25T10:54:31.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b29ffb640 1 -- 172.21.15.19:0/3278768839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035030 con 0x7f6b2c1029c0 2024-04-25T10:54:31.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.126+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1a0ed0 con 0x7f6b2c1029c0 2024-04-25T10:54:31.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.127+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c1a12d0 con 0x7f6b2c1029c0 2024-04-25T10:54:31.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.128+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c107710 con 0x7f6b2c1029c0 2024-04-25T10:54:31.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.129+0000 7f6b29ffb640 1 -- 172.21.15.19:0/3278768839 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f6b14042070 con 0x7f6b2c1029c0 2024-04-25T10:54:31.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.130+0000 7f6b29ffb640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6b180af8e0 0x7f6b180b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.130+0000 7f6b2bfff640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6b180af8e0 0x7f6b180b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.130+0000 7f6b29ffb640 1 -- 172.21.15.19:0/3278768839 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6b140ff1f0 con 0x7f6b2c1029c0 2024-04-25T10:54:31.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.130+0000 7f6b2bfff640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6b180af8e0 0x7f6b180b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c19a1a0 tx=0x7f6b1c009760 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:31.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.135+0000 7f6b29ffb640 1 -- 172.21.15.19:0/3278768839 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b140c7f90 con 0x7f6b2c1029c0 2024-04-25T10:54:31.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.321+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b2c062fe0 con 0x7f6b180af8e0 2024-04-25T10:54:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.329+0000 7f6b29ffb640 1 -- 172.21.15.19:0/3278768839 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257649 (secure 0 0 0) 0x7f6b2c062fe0 con 0x7f6b180af8e0 2024-04-25T10:54:31.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:31.333 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6b180af8e0 msgr2=0x7f6b180b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6b180af8e0 0x7f6b180b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c19a1a0 tx=0x7f6b1c009760 comp rx=0 tx=0).stop 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 msgr2=0x7f6b2c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c199150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6b14031440 tx=0x7f6b14003c10 comp rx=0 tx=0).stop 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 shutdown_connections 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6b180af8e0 0x7f6b180b1df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c108900 0x7f6b2c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b2c103c00 0x7f6b2c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 --2- 172.21.15.19:0/3278768839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b2c1029c0 0x7f6b2c199150 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 >> 172.21.15.19:0/3278768839 conn(0x7f6b2c0fdbd0 msgr2=0x7f6b2c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 shutdown_connections 2024-04-25T10:54:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.332+0000 7f6b32c6e640 1 -- 172.21.15.19:0/3278768839 wait complete. 2024-04-25T10:54:31.344 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":3,"stamp":"2024-04-25T10:54:30.947976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183974,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":419,"num_read_kb":339,"num_write":322,"num_write_kb":3556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366416,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575408,"statfs":{"total":1151940427776,"available":1151565217792,"internally_reserved":0,"allocated":13398016,"data_stored":8022601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729039+0000","last_change":"2024-04-25T10:52:32.474738+0000","last_active":"2024-04-25T10:54:30.729039+0000","last_peered":"2024-04-25T10:54:30.729039+0000","last_clean":"2024-04-25T10:54:30.729039+0000","last_became_active":"2024-04-25T10:52:32.473812+0000","last_became_peered":"2024-04-25T10:52:32.473812+0000","last_unstale":"2024-04-25T10:54:30.729039+0000","last_undegraded":"2024-04-25T10:54:30.729039+0000","last_fullsized":"2024-04-25T10:54:30.729039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:25.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729032+0000","last_change":"2024-04-25T10:52:32.475491+0000","last_active":"2024-04-25T10:54:30.729032+0000","last_peered":"2024-04-25T10:54:30.729032+0000","last_clean":"2024-04-25T10:54:30.729032+0000","last_became_active":"2024-04-25T10:52:32.474164+0000","last_became_peered":"2024-04-25T10:52:32.474164+0000","last_unstale":"2024-04-25T10:54:30.729032+0000","last_undegraded":"2024-04-25T10:54:30.729032+0000","last_fullsized":"2024-04-25T10:54:30.729032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:58.838292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729884+0000","last_change":"2024-04-25T10:52:32.471349+0000","last_active":"2024-04-25T10:54:30.729884+0000","last_peered":"2024-04-25T10:54:30.729884+0000","last_clean":"2024-04-25T10:54:30.729884+0000","last_became_active":"2024-04-25T10:52:32.470972+0000","last_became_peered":"2024-04-25T10:52:32.470972+0000","last_unstale":"2024-04-25T10:54:30.729884+0000","last_undegraded":"2024-04-25T10:54:30.729884+0000","last_fullsized":"2024-04-25T10:54:30.729884+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:28.634537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728967+0000","last_change":"2024-04-25T10:52:32.470212+0000","last_active":"2024-04-25T10:54:30.728967+0000","last_peered":"2024-04-25T10:54:30.728967+0000","last_clean":"2024-04-25T10:54:30.728967+0000","last_became_active":"2024-04-25T10:52:32.469642+0000","last_became_peered":"2024-04-25T10:52:32.469642+0000","last_unstale":"2024-04-25T10:54:30.728967+0000","last_undegraded":"2024-04-25T10:54:30.728967+0000","last_fullsized":"2024-04-25T10:54:30.728967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:08.231679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025284+0000","last_change":"2024-04-25T10:52:32.456568+0000","last_active":"2024-04-25T10:54:30.025284+0000","last_peered":"2024-04-25T10:54:30.025284+0000","last_clean":"2024-04-25T10:54:30.025284+0000","last_became_active":"2024-04-25T10:52:32.456051+0000","last_became_peered":"2024-04-25T10:52:32.456051+0000","last_unstale":"2024-04-25T10:54:30.025284+0000","last_undegraded":"2024-04-25T10:54:30.025284+0000","last_fullsized":"2024-04-25T10:54:30.025284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908129+0000","last_change":"2024-04-25T10:52:32.479781+0000","last_active":"2024-04-25T10:54:29.908129+0000","last_peered":"2024-04-25T10:54:29.908129+0000","last_clean":"2024-04-25T10:54:29.908129+0000","last_became_active":"2024-04-25T10:52:32.479213+0000","last_became_peered":"2024-04-25T10:52:32.479213+0000","last_unstale":"2024-04-25T10:54:29.908129+0000","last_undegraded":"2024-04-25T10:54:29.908129+0000","last_fullsized":"2024-04-25T10:54:29.908129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:16.239969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953981+0000","last_change":"2024-04-25T10:52:32.471430+0000","last_active":"2024-04-25T10:54:29.953981+0000","last_peered":"2024-04-25T10:54:29.953981+0000","last_clean":"2024-04-25T10:54:29.953981+0000","last_became_active":"2024-04-25T10:52:32.470518+0000","last_became_peered":"2024-04-25T10:52:32.470518+0000","last_unstale":"2024-04-25T10:54:29.953981+0000","last_undegraded":"2024-04-25T10:54:29.953981+0000","last_fullsized":"2024-04-25T10:54:29.953981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:12.846174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.906963+0000","last_change":"2024-04-25T10:52:32.472545+0000","last_active":"2024-04-25T10:54:29.906963+0000","last_peered":"2024-04-25T10:54:29.906963+0000","last_clean":"2024-04-25T10:54:29.906963+0000","last_became_active":"2024-04-25T10:52:32.472151+0000","last_became_peered":"2024-04-25T10:52:32.472151+0000","last_unstale":"2024-04-25T10:54:29.906963+0000","last_undegraded":"2024-04-25T10:54:29.906963+0000","last_fullsized":"2024-04-25T10:54:29.906963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:36.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.214026+0000","last_change":"2024-04-25T10:52:32.466223+0000","last_active":"2024-04-25T10:54:30.214026+0000","last_peered":"2024-04-25T10:54:30.214026+0000","last_clean":"2024-04-25T10:54:30.214026+0000","last_became_active":"2024-04-25T10:52:32.465498+0000","last_became_peered":"2024-04-25T10:52:32.465498+0000","last_unstale":"2024-04-25T10:54:30.214026+0000","last_undegraded":"2024-04-25T10:54:30.214026+0000","last_fullsized":"2024-04-25T10:54:30.214026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:25.924450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908452+0000","last_change":"2024-04-25T10:52:32.470601+0000","last_active":"2024-04-25T10:54:29.908452+0000","last_peered":"2024-04-25T10:54:29.908452+0000","last_clean":"2024-04-25T10:54:29.908452+0000","last_became_active":"2024-04-25T10:52:32.468984+0000","last_became_peered":"2024-04-25T10:52:32.468984+0000","last_unstale":"2024-04-25T10:54:29.908452+0000","last_undegraded":"2024-04-25T10:54:29.908452+0000","last_fullsized":"2024-04-25T10:54:29.908452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954302+0000","last_change":"2024-04-25T10:52:32.463907+0000","last_active":"2024-04-25T10:54:29.954302+0000","last_peered":"2024-04-25T10:54:29.954302+0000","last_clean":"2024-04-25T10:54:29.954302+0000","last_became_active":"2024-04-25T10:52:32.463536+0000","last_became_peered":"2024-04-25T10:52:32.463536+0000","last_unstale":"2024-04-25T10:54:29.954302+0000","last_undegraded":"2024-04-25T10:54:29.954302+0000","last_fullsized":"2024-04-25T10:54:29.954302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.503805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728733+0000","last_change":"2024-04-25T10:52:32.473646+0000","last_active":"2024-04-25T10:54:30.728733+0000","last_peered":"2024-04-25T10:54:30.728733+0000","last_clean":"2024-04-25T10:54:30.728733+0000","last_became_active":"2024-04-25T10:52:32.473310+0000","last_became_peered":"2024-04-25T10:52:32.473310+0000","last_unstale":"2024-04-25T10:54:30.728733+0000","last_undegraded":"2024-04-25T10:54:30.728733+0000","last_fullsized":"2024-04-25T10:54:30.728733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.691293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728931+0000","last_change":"2024-04-25T10:53:31.244988+0000","last_active":"2024-04-25T10:54:30.728931+0000","last_peered":"2024-04-25T10:54:30.728931+0000","last_clean":"2024-04-25T10:54:30.728931+0000","last_became_active":"2024-04-25T10:53:31.244535+0000","last_became_peered":"2024-04-25T10:53:31.244535+0000","last_unstale":"2024-04-25T10:54:30.728931+0000","last_undegraded":"2024-04-25T10:54:30.728931+0000","last_fullsized":"2024-04-25T10:54:30.728931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:14.205691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729478+0000","last_change":"2024-04-25T10:52:32.476104+0000","last_active":"2024-04-25T10:54:30.729478+0000","last_peered":"2024-04-25T10:54:30.729478+0000","last_clean":"2024-04-25T10:54:30.729478+0000","last_became_active":"2024-04-25T10:52:32.475342+0000","last_became_peered":"2024-04-25T10:52:32.475342+0000","last_unstale":"2024-04-25T10:54:30.729478+0000","last_undegraded":"2024-04-25T10:54:30.729478+0000","last_fullsized":"2024-04-25T10:54:30.729478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:37.087793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729220+0000","last_change":"2024-04-25T10:52:32.470263+0000","last_active":"2024-04-25T10:54:30.729220+0000","last_peered":"2024-04-25T10:54:30.729220+0000","last_clean":"2024-04-25T10:54:30.729220+0000","last_became_active":"2024-04-25T10:52:32.469873+0000","last_became_peered":"2024-04-25T10:52:32.469873+0000","last_unstale":"2024-04-25T10:54:30.729220+0000","last_undegraded":"2024-04-25T10:54:30.729220+0000","last_fullsized":"2024-04-25T10:54:30.729220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.349598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213526+0000","last_change":"2024-04-25T10:52:32.471851+0000","last_active":"2024-04-25T10:54:30.213526+0000","last_peered":"2024-04-25T10:54:30.213526+0000","last_clean":"2024-04-25T10:54:30.213526+0000","last_became_active":"2024-04-25T10:52:32.471477+0000","last_became_peered":"2024-04-25T10:52:32.471477+0000","last_unstale":"2024-04-25T10:54:30.213526+0000","last_undegraded":"2024-04-25T10:54:30.213526+0000","last_fullsized":"2024-04-25T10:54:30.213526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:59.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728928+0000","last_change":"2024-04-25T10:52:32.472530+0000","last_active":"2024-04-25T10:54:30.728928+0000","last_peered":"2024-04-25T10:54:30.728928+0000","last_clean":"2024-04-25T10:54:30.728928+0000","last_became_active":"2024-04-25T10:52:32.472174+0000","last_became_peered":"2024-04-25T10:52:32.472174+0000","last_unstale":"2024-04-25T10:54:30.728928+0000","last_undegraded":"2024-04-25T10:54:30.728928+0000","last_fullsized":"2024-04-25T10:54:30.728928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:09.473575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022991+0000","last_change":"2024-04-25T10:52:32.473617+0000","last_active":"2024-04-25T10:54:30.022991+0000","last_peered":"2024-04-25T10:54:30.022991+0000","last_clean":"2024-04-25T10:54:30.022991+0000","last_became_active":"2024-04-25T10:52:32.473261+0000","last_became_peered":"2024-04-25T10:52:32.473261+0000","last_unstale":"2024-04-25T10:54:30.022991+0000","last_undegraded":"2024-04-25T10:54:30.022991+0000","last_fullsized":"2024-04-25T10:54:30.022991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:05.568672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213897+0000","last_change":"2024-04-25T10:52:32.466296+0000","last_active":"2024-04-25T10:54:30.213897+0000","last_peered":"2024-04-25T10:54:30.213897+0000","last_clean":"2024-04-25T10:54:30.213897+0000","last_became_active":"2024-04-25T10:52:32.463153+0000","last_became_peered":"2024-04-25T10:52:32.463153+0000","last_unstale":"2024-04-25T10:54:30.213897+0000","last_undegraded":"2024-04-25T10:54:30.213897+0000","last_fullsized":"2024-04-25T10:54:30.213897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022989+0000","last_change":"2024-04-25T10:52:31.701949+0000","last_active":"2024-04-25T10:54:30.022989+0000","last_peered":"2024-04-25T10:54:30.022989+0000","last_clean":"2024-04-25T10:54:30.022989+0000","last_became_active":"2024-04-25T10:52:31.701508+0000","last_became_peered":"2024-04-25T10:52:31.701508+0000","last_unstale":"2024-04-25T10:54:30.022989+0000","last_undegraded":"2024-04-25T10:54:30.022989+0000","last_fullsized":"2024-04-25T10:54:30.022989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:41.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907167+0000","last_change":"2024-04-25T10:52:32.468871+0000","last_active":"2024-04-25T10:54:29.907167+0000","last_peered":"2024-04-25T10:54:29.907167+0000","last_clean":"2024-04-25T10:54:29.907167+0000","last_became_active":"2024-04-25T10:52:32.468460+0000","last_became_peered":"2024-04-25T10:52:32.468460+0000","last_unstale":"2024-04-25T10:54:29.907167+0000","last_undegraded":"2024-04-25T10:54:29.907167+0000","last_fullsized":"2024-04-25T10:54:29.907167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:29.055639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907643+0000","last_change":"2024-04-25T10:52:31.705068+0000","last_active":"2024-04-25T10:54:29.907643+0000","last_peered":"2024-04-25T10:54:29.907643+0000","last_clean":"2024-04-25T10:54:29.907643+0000","last_became_active":"2024-04-25T10:52:31.704439+0000","last_became_peered":"2024-04-25T10:52:31.704439+0000","last_unstale":"2024-04-25T10:54:29.907643+0000","last_undegraded":"2024-04-25T10:54:29.907643+0000","last_fullsized":"2024-04-25T10:54:29.907643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.347653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729948+0000","last_change":"2024-04-25T10:52:32.476025+0000","last_active":"2024-04-25T10:54:30.729948+0000","last_peered":"2024-04-25T10:54:30.729948+0000","last_clean":"2024-04-25T10:54:30.729948+0000","last_became_active":"2024-04-25T10:52:32.475611+0000","last_became_peered":"2024-04-25T10:52:32.475611+0000","last_unstale":"2024-04-25T10:54:30.729948+0000","last_undegraded":"2024-04-25T10:54:30.729948+0000","last_fullsized":"2024-04-25T10:54:30.729948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:45.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025643+0000","last_change":"2024-04-25T10:52:31.703012+0000","last_active":"2024-04-25T10:54:30.025643+0000","last_peered":"2024-04-25T10:54:30.025643+0000","last_clean":"2024-04-25T10:54:30.025643+0000","last_became_active":"2024-04-25T10:52:31.702689+0000","last_became_peered":"2024-04-25T10:52:31.702689+0000","last_unstale":"2024-04-25T10:54:30.025643+0000","last_undegraded":"2024-04-25T10:54:30.025643+0000","last_fullsized":"2024-04-25T10:54:30.025643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:13.038859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213642+0000","last_change":"2024-04-25T10:52:32.475858+0000","last_active":"2024-04-25T10:54:30.213642+0000","last_peered":"2024-04-25T10:54:30.213642+0000","last_clean":"2024-04-25T10:54:30.213642+0000","last_became_active":"2024-04-25T10:52:32.475486+0000","last_became_peered":"2024-04-25T10:52:32.475486+0000","last_unstale":"2024-04-25T10:54:30.213642+0000","last_undegraded":"2024-04-25T10:54:30.213642+0000","last_fullsized":"2024-04-25T10:54:30.213642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.439208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728623+0000","last_change":"2024-04-25T10:53:31.155911+0000","last_active":"2024-04-25T10:54:30.728623+0000","last_peered":"2024-04-25T10:54:30.728623+0000","last_clean":"2024-04-25T10:54:30.728623+0000","last_became_active":"2024-04-25T10:53:31.151778+0000","last_became_peered":"2024-04-25T10:53:31.151778+0000","last_unstale":"2024-04-25T10:54:30.728623+0000","last_undegraded":"2024-04-25T10:54:30.728623+0000","last_fullsized":"2024-04-25T10:54:30.728623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:46.936583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908530+0000","last_change":"2024-04-25T10:53:31.152237+0000","last_active":"2024-04-25T10:54:29.908530+0000","last_peered":"2024-04-25T10:54:29.908530+0000","last_clean":"2024-04-25T10:54:29.908530+0000","last_became_active":"2024-04-25T10:53:31.151821+0000","last_became_peered":"2024-04-25T10:53:31.151821+0000","last_unstale":"2024-04-25T10:54:29.908530+0000","last_undegraded":"2024-04-25T10:54:29.908530+0000","last_fullsized":"2024-04-25T10:54:29.908530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:02.453018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908200+0000","last_change":"2024-04-25T10:52:31.699644+0000","last_active":"2024-04-25T10:54:29.908200+0000","last_peered":"2024-04-25T10:54:29.908200+0000","last_clean":"2024-04-25T10:54:29.908200+0000","last_became_active":"2024-04-25T10:52:31.699315+0000","last_became_peered":"2024-04-25T10:52:31.699315+0000","last_unstale":"2024-04-25T10:54:29.908200+0000","last_undegraded":"2024-04-25T10:54:29.908200+0000","last_fullsized":"2024-04-25T10:54:29.908200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:52.915096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025139+0000","last_change":"2024-04-25T10:52:32.463372+0000","last_active":"2024-04-25T10:54:30.025139+0000","last_peered":"2024-04-25T10:54:30.025139+0000","last_clean":"2024-04-25T10:54:30.025139+0000","last_became_active":"2024-04-25T10:52:32.462926+0000","last_became_peered":"2024-04-25T10:52:32.462926+0000","last_unstale":"2024-04-25T10:54:30.025139+0000","last_undegraded":"2024-04-25T10:54:30.025139+0000","last_fullsized":"2024-04-25T10:54:30.025139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:17.724268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908048+0000","last_change":"2024-04-25T10:52:31.695754+0000","last_active":"2024-04-25T10:54:29.908048+0000","last_peered":"2024-04-25T10:54:29.908048+0000","last_clean":"2024-04-25T10:54:29.908048+0000","last_became_active":"2024-04-25T10:52:31.695381+0000","last_became_peered":"2024-04-25T10:52:31.695381+0000","last_unstale":"2024-04-25T10:54:29.908048+0000","last_undegraded":"2024-04-25T10:54:29.908048+0000","last_fullsized":"2024-04-25T10:54:29.908048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:58.851534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908339+0000","last_change":"2024-04-25T10:53:31.154575+0000","last_active":"2024-04-25T10:54:29.908339+0000","last_peered":"2024-04-25T10:54:29.908339+0000","last_clean":"2024-04-25T10:54:29.908339+0000","last_became_active":"2024-04-25T10:53:31.154145+0000","last_became_peered":"2024-04-25T10:53:31.154145+0000","last_unstale":"2024-04-25T10:54:29.908339+0000","last_undegraded":"2024-04-25T10:54:29.908339+0000","last_fullsized":"2024-04-25T10:54:29.908339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:37.838587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728773+0000","last_change":"2024-04-25T10:52:31.695067+0000","last_active":"2024-04-25T10:54:30.728773+0000","last_peered":"2024-04-25T10:54:30.728773+0000","last_clean":"2024-04-25T10:54:30.728773+0000","last_became_active":"2024-04-25T10:52:31.694648+0000","last_became_peered":"2024-04-25T10:52:31.694648+0000","last_unstale":"2024-04-25T10:54:30.728773+0000","last_undegraded":"2024-04-25T10:54:30.728773+0000","last_fullsized":"2024-04-25T10:54:30.728773+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:09.654835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907056+0000","last_change":"2024-04-25T10:52:32.453501+0000","last_active":"2024-04-25T10:54:29.907056+0000","last_peered":"2024-04-25T10:54:29.907056+0000","last_clean":"2024-04-25T10:54:29.907056+0000","last_became_active":"2024-04-25T10:52:32.453113+0000","last_became_peered":"2024-04-25T10:52:32.453113+0000","last_unstale":"2024-04-25T10:54:29.907056+0000","last_undegraded":"2024-04-25T10:54:29.907056+0000","last_fullsized":"2024-04-25T10:54:29.907056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:07.340709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908088+0000","last_change":"2024-04-25T10:52:31.697946+0000","last_active":"2024-04-25T10:54:29.908088+0000","last_peered":"2024-04-25T10:54:29.908088+0000","last_clean":"2024-04-25T10:54:29.908088+0000","last_became_active":"2024-04-25T10:52:31.697612+0000","last_became_peered":"2024-04-25T10:52:31.697612+0000","last_unstale":"2024-04-25T10:54:29.908088+0000","last_undegraded":"2024-04-25T10:54:29.908088+0000","last_fullsized":"2024-04-25T10:54:29.908088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.280776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022464+0000","last_change":"2024-04-25T10:52:32.473537+0000","last_active":"2024-04-25T10:54:30.022464+0000","last_peered":"2024-04-25T10:54:30.022464+0000","last_clean":"2024-04-25T10:54:30.022464+0000","last_became_active":"2024-04-25T10:52:32.473164+0000","last_became_peered":"2024-04-25T10:52:32.473164+0000","last_unstale":"2024-04-25T10:54:30.022464+0000","last_undegraded":"2024-04-25T10:54:30.022464+0000","last_fullsized":"2024-04-25T10:54:30.022464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.008031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908381+0000","last_change":"2024-04-25T10:52:31.694207+0000","last_active":"2024-04-25T10:54:29.908381+0000","last_peered":"2024-04-25T10:54:29.908381+0000","last_clean":"2024-04-25T10:54:29.908381+0000","last_became_active":"2024-04-25T10:52:31.693777+0000","last_became_peered":"2024-04-25T10:52:31.693777+0000","last_unstale":"2024-04-25T10:54:29.908381+0000","last_undegraded":"2024-04-25T10:54:29.908381+0000","last_fullsized":"2024-04-25T10:54:29.908381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:43.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213960+0000","last_change":"2024-04-25T10:52:32.475545+0000","last_active":"2024-04-25T10:54:30.213960+0000","last_peered":"2024-04-25T10:54:30.213960+0000","last_clean":"2024-04-25T10:54:30.213960+0000","last_became_active":"2024-04-25T10:52:32.475195+0000","last_became_peered":"2024-04-25T10:52:32.475195+0000","last_unstale":"2024-04-25T10:54:30.213960+0000","last_undegraded":"2024-04-25T10:54:30.213960+0000","last_fullsized":"2024-04-25T10:54:30.213960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:49.900374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954240+0000","last_change":"2024-04-25T10:52:31.698184+0000","last_active":"2024-04-25T10:54:29.954240+0000","last_peered":"2024-04-25T10:54:29.954240+0000","last_clean":"2024-04-25T10:54:29.954240+0000","last_became_active":"2024-04-25T10:52:31.697816+0000","last_became_peered":"2024-04-25T10:52:31.697816+0000","last_unstale":"2024-04-25T10:54:29.954240+0000","last_undegraded":"2024-04-25T10:54:29.954240+0000","last_fullsized":"2024-04-25T10:54:29.954240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:44.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729953+0000","last_change":"2024-04-25T10:52:32.472437+0000","last_active":"2024-04-25T10:54:30.729953+0000","last_peered":"2024-04-25T10:54:30.729953+0000","last_clean":"2024-04-25T10:54:30.729953+0000","last_became_active":"2024-04-25T10:52:32.471944+0000","last_became_peered":"2024-04-25T10:52:32.471944+0000","last_unstale":"2024-04-25T10:54:30.729953+0000","last_undegraded":"2024-04-25T10:54:30.729953+0000","last_fullsized":"2024-04-25T10:54:30.729953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:11.047158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908612+0000","last_change":"2024-04-25T10:52:31.696943+0000","last_active":"2024-04-25T10:54:29.908612+0000","last_peered":"2024-04-25T10:54:29.908612+0000","last_clean":"2024-04-25T10:54:29.908612+0000","last_became_active":"2024-04-25T10:52:31.696561+0000","last_became_peered":"2024-04-25T10:52:31.696561+0000","last_unstale":"2024-04-25T10:54:29.908612+0000","last_undegraded":"2024-04-25T10:54:29.908612+0000","last_fullsized":"2024-04-25T10:54:29.908612+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:14.628800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213863+0000","last_change":"2024-04-25T10:52:32.461966+0000","last_active":"2024-04-25T10:54:30.213863+0000","last_peered":"2024-04-25T10:54:30.213863+0000","last_clean":"2024-04-25T10:54:30.213863+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:54:30.213863+0000","last_undegraded":"2024-04-25T10:54:30.213863+0000","last_fullsized":"2024-04-25T10:54:30.213863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":295,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213415+0000","last_change":"2024-04-25T10:52:31.690938+0000","last_active":"2024-04-25T10:54:30.213415+0000","last_peered":"2024-04-25T10:54:30.213415+0000","last_clean":"2024-04-25T10:54:30.213415+0000","last_became_active":"2024-04-25T10:52:31.690511+0000","last_became_peered":"2024-04-25T10:52:31.690511+0000","last_unstale":"2024-04-25T10:54:30.213415+0000","last_undegraded":"2024-04-25T10:54:30.213415+0000","last_fullsized":"2024-04-25T10:54:30.213415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:10.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":240,"num_read_kb":185,"num_write":34,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025452+0000","last_change":"2024-04-25T10:52:32.474871+0000","last_active":"2024-04-25T10:54:30.025452+0000","last_peered":"2024-04-25T10:54:30.025452+0000","last_clean":"2024-04-25T10:54:30.025452+0000","last_became_active":"2024-04-25T10:52:32.474463+0000","last_became_peered":"2024-04-25T10:52:32.474463+0000","last_unstale":"2024-04-25T10:54:30.025452+0000","last_undegraded":"2024-04-25T10:54:30.025452+0000","last_fullsized":"2024-04-25T10:54:30.025452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:36.719230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213752+0000","last_change":"2024-04-25T10:52:31.691006+0000","last_active":"2024-04-25T10:54:30.213752+0000","last_peered":"2024-04-25T10:54:30.213752+0000","last_clean":"2024-04-25T10:54:30.213752+0000","last_became_active":"2024-04-25T10:52:31.690611+0000","last_became_peered":"2024-04-25T10:52:31.690611+0000","last_unstale":"2024-04-25T10:54:30.213752+0000","last_undegraded":"2024-04-25T10:54:30.213752+0000","last_fullsized":"2024-04-25T10:54:30.213752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:37.781160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728620+0000","last_change":"2024-04-25T10:52:32.463331+0000","last_active":"2024-04-25T10:54:30.728620+0000","last_peered":"2024-04-25T10:54:30.728620+0000","last_clean":"2024-04-25T10:54:30.728620+0000","last_became_active":"2024-04-25T10:52:32.462579+0000","last_became_peered":"2024-04-25T10:52:32.462579+0000","last_unstale":"2024-04-25T10:54:30.728620+0000","last_undegraded":"2024-04-25T10:54:30.728620+0000","last_fullsized":"2024-04-25T10:54:30.728620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908256+0000","last_change":"2024-04-25T10:52:31.704675+0000","last_active":"2024-04-25T10:54:29.908256+0000","last_peered":"2024-04-25T10:54:29.908256+0000","last_clean":"2024-04-25T10:54:29.908256+0000","last_became_active":"2024-04-25T10:52:31.704236+0000","last_became_peered":"2024-04-25T10:52:31.704236+0000","last_unstale":"2024-04-25T10:54:29.908256+0000","last_undegraded":"2024-04-25T10:54:29.908256+0000","last_fullsized":"2024-04-25T10:54:29.908256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.637006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954166+0000","last_change":"2024-04-25T10:52:32.462960+0000","last_active":"2024-04-25T10:54:29.954166+0000","last_peered":"2024-04-25T10:54:29.954166+0000","last_clean":"2024-04-25T10:54:29.954166+0000","last_became_active":"2024-04-25T10:52:32.461993+0000","last_became_peered":"2024-04-25T10:52:32.461993+0000","last_unstale":"2024-04-25T10:54:29.954166+0000","last_undegraded":"2024-04-25T10:54:29.954166+0000","last_fullsized":"2024-04-25T10:54:29.954166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:01.803209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728552+0000","last_change":"2024-04-25T10:52:31.698206+0000","last_active":"2024-04-25T10:54:30.728552+0000","last_peered":"2024-04-25T10:54:30.728552+0000","last_clean":"2024-04-25T10:54:30.728552+0000","last_became_active":"2024-04-25T10:52:31.697814+0000","last_became_peered":"2024-04-25T10:52:31.697814+0000","last_unstale":"2024-04-25T10:54:30.728552+0000","last_undegraded":"2024-04-25T10:54:30.728552+0000","last_fullsized":"2024-04-25T10:54:30.728552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:02.849229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953813+0000","last_change":"2024-04-25T10:52:32.473551+0000","last_active":"2024-04-25T10:54:29.953813+0000","last_peered":"2024-04-25T10:54:29.953813+0000","last_clean":"2024-04-25T10:54:29.953813+0000","last_became_active":"2024-04-25T10:52:32.473227+0000","last_became_peered":"2024-04-25T10:52:32.473227+0000","last_unstale":"2024-04-25T10:54:29.953813+0000","last_undegraded":"2024-04-25T10:54:29.953813+0000","last_fullsized":"2024-04-25T10:54:29.953813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:37.742066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025114+0000","last_change":"2024-04-25T10:52:31.700670+0000","last_active":"2024-04-25T10:54:30.025114+0000","last_peered":"2024-04-25T10:54:30.025114+0000","last_clean":"2024-04-25T10:54:30.025114+0000","last_became_active":"2024-04-25T10:52:31.700234+0000","last_became_peered":"2024-04-25T10:52:31.700234+0000","last_unstale":"2024-04-25T10:54:30.025114+0000","last_undegraded":"2024-04-25T10:54:30.025114+0000","last_fullsized":"2024-04-25T10:54:30.025114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:21.795784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908220+0000","last_change":"2024-04-25T10:53:31.153193+0000","last_active":"2024-04-25T10:54:29.908220+0000","last_peered":"2024-04-25T10:54:29.908220+0000","last_clean":"2024-04-25T10:54:29.908220+0000","last_became_active":"2024-04-25T10:53:31.152723+0000","last_became_peered":"2024-04-25T10:53:31.152723+0000","last_unstale":"2024-04-25T10:54:29.908220+0000","last_undegraded":"2024-04-25T10:54:29.908220+0000","last_fullsized":"2024-04-25T10:54:29.908220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:42.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025555+0000","last_change":"2024-04-25T10:52:31.698507+0000","last_active":"2024-04-25T10:54:30.025555+0000","last_peered":"2024-04-25T10:54:30.025555+0000","last_clean":"2024-04-25T10:54:30.025555+0000","last_became_active":"2024-04-25T10:52:31.698148+0000","last_became_peered":"2024-04-25T10:52:31.698148+0000","last_unstale":"2024-04-25T10:54:30.025555+0000","last_undegraded":"2024-04-25T10:54:30.025555+0000","last_fullsized":"2024-04-25T10:54:30.025555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:48.993234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728798+0000","last_change":"2024-04-25T10:52:32.465874+0000","last_active":"2024-04-25T10:54:30.728798+0000","last_peered":"2024-04-25T10:54:30.728798+0000","last_clean":"2024-04-25T10:54:30.728798+0000","last_became_active":"2024-04-25T10:52:32.465511+0000","last_became_peered":"2024-04-25T10:52:32.465511+0000","last_unstale":"2024-04-25T10:54:30.728798+0000","last_undegraded":"2024-04-25T10:54:30.728798+0000","last_fullsized":"2024-04-25T10:54:30.728798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:15.274909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022791+0000","last_change":"2024-04-25T10:52:31.691074+0000","last_active":"2024-04-25T10:54:30.022791+0000","last_peered":"2024-04-25T10:54:30.022791+0000","last_clean":"2024-04-25T10:54:30.022791+0000","last_became_active":"2024-04-25T10:52:31.690402+0000","last_became_peered":"2024-04-25T10:52:31.690402+0000","last_unstale":"2024-04-25T10:54:30.022791+0000","last_undegraded":"2024-04-25T10:54:30.022791+0000","last_fullsized":"2024-04-25T10:54:30.022791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.529860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908342+0000","last_change":"2024-04-25T10:52:32.477968+0000","last_active":"2024-04-25T10:54:29.908342+0000","last_peered":"2024-04-25T10:54:29.908342+0000","last_clean":"2024-04-25T10:54:29.908342+0000","last_became_active":"2024-04-25T10:52:32.476395+0000","last_became_peered":"2024-04-25T10:52:32.476395+0000","last_unstale":"2024-04-25T10:54:29.908342+0000","last_undegraded":"2024-04-25T10:54:29.908342+0000","last_fullsized":"2024-04-25T10:54:29.908342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:13.128910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953969+0000","last_change":"2024-04-25T10:52:31.698934+0000","last_active":"2024-04-25T10:54:29.953969+0000","last_peered":"2024-04-25T10:54:29.953969+0000","last_clean":"2024-04-25T10:54:29.953969+0000","last_became_active":"2024-04-25T10:52:31.698565+0000","last_became_peered":"2024-04-25T10:52:31.698565+0000","last_unstale":"2024-04-25T10:54:29.953969+0000","last_undegraded":"2024-04-25T10:54:29.953969+0000","last_fullsized":"2024-04-25T10:54:29.953969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.360617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908305+0000","last_change":"2024-04-25T10:52:32.477538+0000","last_active":"2024-04-25T10:54:29.908305+0000","last_peered":"2024-04-25T10:54:29.908305+0000","last_clean":"2024-04-25T10:54:29.908305+0000","last_became_active":"2024-04-25T10:52:32.476088+0000","last_became_peered":"2024-04-25T10:52:32.476088+0000","last_unstale":"2024-04-25T10:54:29.908305+0000","last_undegraded":"2024-04-25T10:54:29.908305+0000","last_fullsized":"2024-04-25T10:54:29.908305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.987071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022333+0000","last_change":"2024-04-25T10:53:31.244447+0000","last_active":"2024-04-25T10:54:30.022333+0000","last_peered":"2024-04-25T10:54:30.022333+0000","last_clean":"2024-04-25T10:54:30.022333+0000","last_became_active":"2024-04-25T10:53:31.243944+0000","last_became_peered":"2024-04-25T10:53:31.243944+0000","last_unstale":"2024-04-25T10:54:30.022333+0000","last_undegraded":"2024-04-25T10:54:30.022333+0000","last_fullsized":"2024-04-25T10:54:30.022333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:46.344278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728666+0000","last_change":"2024-04-25T10:52:31.706700+0000","last_active":"2024-04-25T10:54:30.728666+0000","last_peered":"2024-04-25T10:54:30.728666+0000","last_clean":"2024-04-25T10:54:30.728666+0000","last_became_active":"2024-04-25T10:52:31.706392+0000","last_became_peered":"2024-04-25T10:52:31.706392+0000","last_unstale":"2024-04-25T10:54:30.728666+0000","last_undegraded":"2024-04-25T10:54:30.728666+0000","last_fullsized":"2024-04-25T10:54:30.728666+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:58.867351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907365+0000","last_change":"2024-04-25T10:52:32.462603+0000","last_active":"2024-04-25T10:54:29.907365+0000","last_peered":"2024-04-25T10:54:29.907365+0000","last_clean":"2024-04-25T10:54:29.907365+0000","last_became_active":"2024-04-25T10:52:32.462238+0000","last_became_peered":"2024-04-25T10:52:32.462238+0000","last_unstale":"2024-04-25T10:54:29.907365+0000","last_undegraded":"2024-04-25T10:54:29.907365+0000","last_fullsized":"2024-04-25T10:54:29.907365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.555683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728675+0000","last_change":"2024-04-25T10:52:31.694054+0000","last_active":"2024-04-25T10:54:30.728675+0000","last_peered":"2024-04-25T10:54:30.728675+0000","last_clean":"2024-04-25T10:54:30.728675+0000","last_became_active":"2024-04-25T10:52:31.693693+0000","last_became_peered":"2024-04-25T10:52:31.693693+0000","last_unstale":"2024-04-25T10:54:30.728675+0000","last_undegraded":"2024-04-25T10:54:30.728675+0000","last_fullsized":"2024-04-25T10:54:30.728675+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:22.238131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953885+0000","last_change":"2024-04-25T10:51:22.730925+0000","last_active":"2024-04-25T10:54:29.953885+0000","last_peered":"2024-04-25T10:54:29.953885+0000","last_clean":"2024-04-25T10:54:29.953885+0000","last_became_active":"2024-04-25T10:51:22.725168+0000","last_became_peered":"2024-04-25T10:51:22.725168+0000","last_unstale":"2024-04-25T10:54:29.953885+0000","last_undegraded":"2024-04-25T10:54:29.953885+0000","last_fullsized":"2024-04-25T10:54:29.953885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_clean_scrub_stamp":"2024-04-25T10:47:56.758246+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:02.416333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728494+0000","last_change":"2024-04-25T10:52:32.461608+0000","last_active":"2024-04-25T10:54:30.728494+0000","last_peered":"2024-04-25T10:54:30.728494+0000","last_clean":"2024-04-25T10:54:30.728494+0000","last_became_active":"2024-04-25T10:52:32.461210+0000","last_became_peered":"2024-04-25T10:52:32.461210+0000","last_unstale":"2024-04-25T10:54:30.728494+0000","last_undegraded":"2024-04-25T10:54:30.728494+0000","last_fullsized":"2024-04-25T10:54:30.728494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:14.897053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954025+0000","last_change":"2024-04-25T10:52:31.701003+0000","last_active":"2024-04-25T10:54:29.954025+0000","last_peered":"2024-04-25T10:54:29.954025+0000","last_clean":"2024-04-25T10:54:29.954025+0000","last_became_active":"2024-04-25T10:52:31.699623+0000","last_became_peered":"2024-04-25T10:52:31.699623+0000","last_unstale":"2024-04-25T10:54:29.954025+0000","last_undegraded":"2024-04-25T10:54:29.954025+0000","last_fullsized":"2024-04-25T10:54:29.954025+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:45.175760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025523+0000","last_change":"2024-04-25T10:52:31.702595+0000","last_active":"2024-04-25T10:54:30.025523+0000","last_peered":"2024-04-25T10:54:30.025523+0000","last_clean":"2024-04-25T10:54:30.025523+0000","last_became_active":"2024-04-25T10:52:31.702124+0000","last_became_peered":"2024-04-25T10:52:31.702124+0000","last_unstale":"2024-04-25T10:54:30.025523+0000","last_undegraded":"2024-04-25T10:54:30.025523+0000","last_fullsized":"2024-04-25T10:54:30.025523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:54.351859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728982+0000","last_change":"2024-04-25T10:52:32.467987+0000","last_active":"2024-04-25T10:54:30.728982+0000","last_peered":"2024-04-25T10:54:30.728982+0000","last_clean":"2024-04-25T10:54:30.728982+0000","last_became_active":"2024-04-25T10:52:32.467613+0000","last_became_peered":"2024-04-25T10:52:32.467613+0000","last_unstale":"2024-04-25T10:54:30.728982+0000","last_undegraded":"2024-04-25T10:54:30.728982+0000","last_fullsized":"2024-04-25T10:54:30.728982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:00.112102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'43","reported_seq":87,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.159606+0000","last_change":"2024-04-25T10:52:31.699868+0000","last_active":"2024-04-25T10:54:30.159606+0000","last_peered":"2024-04-25T10:54:30.159606+0000","last_clean":"2024-04-25T10:54:30.159606+0000","last_became_active":"2024-04-25T10:52:31.699470+0000","last_became_peered":"2024-04-25T10:52:31.699470+0000","last_unstale":"2024-04-25T10:54:30.159606+0000","last_undegraded":"2024-04-25T10:54:30.159606+0000","last_fullsized":"2024-04-25T10:54:30.159606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.161821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":77,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907687+0000","last_change":"2024-04-25T10:52:32.459082+0000","last_active":"2024-04-25T10:54:29.907687+0000","last_peered":"2024-04-25T10:54:29.907687+0000","last_clean":"2024-04-25T10:54:29.907687+0000","last_became_active":"2024-04-25T10:52:32.458190+0000","last_became_peered":"2024-04-25T10:52:32.458190+0000","last_unstale":"2024-04-25T10:54:29.907687+0000","last_undegraded":"2024-04-25T10:54:29.907687+0000","last_fullsized":"2024-04-25T10:54:29.907687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.280762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728730+0000","last_change":"2024-04-25T10:52:31.700636+0000","last_active":"2024-04-25T10:54:30.728730+0000","last_peered":"2024-04-25T10:54:30.728730+0000","last_clean":"2024-04-25T10:54:30.728730+0000","last_became_active":"2024-04-25T10:52:31.700268+0000","last_became_peered":"2024-04-25T10:52:31.700268+0000","last_unstale":"2024-04-25T10:54:30.728730+0000","last_undegraded":"2024-04-25T10:54:30.728730+0000","last_fullsized":"2024-04-25T10:54:30.728730+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.439491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729188+0000","last_change":"2024-04-25T10:52:32.467406+0000","last_active":"2024-04-25T10:54:30.729188+0000","last_peered":"2024-04-25T10:54:30.729188+0000","last_clean":"2024-04-25T10:54:30.729188+0000","last_became_active":"2024-04-25T10:52:32.467053+0000","last_became_peered":"2024-04-25T10:52:32.467053+0000","last_unstale":"2024-04-25T10:54:30.729188+0000","last_undegraded":"2024-04-25T10:54:30.729188+0000","last_fullsized":"2024-04-25T10:54:30.729188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213604+0000","last_change":"2024-04-25T10:53:31.301411+0000","last_active":"2024-04-25T10:54:30.213604+0000","last_peered":"2024-04-25T10:54:30.213604+0000","last_clean":"2024-04-25T10:54:30.213604+0000","last_became_active":"2024-04-25T10:53:31.301022+0000","last_became_peered":"2024-04-25T10:53:31.301022+0000","last_unstale":"2024-04-25T10:54:30.213604+0000","last_undegraded":"2024-04-25T10:54:30.213604+0000","last_fullsized":"2024-04-25T10:54:30.213604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:28.171444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025114+0000","last_change":"2024-04-25T10:52:32.472709+0000","last_active":"2024-04-25T10:54:30.025114+0000","last_peered":"2024-04-25T10:54:30.025114+0000","last_clean":"2024-04-25T10:54:30.025114+0000","last_became_active":"2024-04-25T10:52:32.472331+0000","last_became_peered":"2024-04-25T10:52:32.472331+0000","last_unstale":"2024-04-25T10:54:30.025114+0000","last_undegraded":"2024-04-25T10:54:30.025114+0000","last_fullsized":"2024-04-25T10:54:30.025114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.047682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907099+0000","last_change":"2024-04-25T10:52:31.697360+0000","last_active":"2024-04-25T10:54:29.907099+0000","last_peered":"2024-04-25T10:54:29.907099+0000","last_clean":"2024-04-25T10:54:29.907099+0000","last_became_active":"2024-04-25T10:52:31.696991+0000","last_became_peered":"2024-04-25T10:52:31.696991+0000","last_unstale":"2024-04-25T10:54:29.907099+0000","last_undegraded":"2024-04-25T10:54:29.907099+0000","last_fullsized":"2024-04-25T10:54:29.907099+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:38.395372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728660+0000","last_change":"2024-04-25T10:52:32.472505+0000","last_active":"2024-04-25T10:54:30.728660+0000","last_peered":"2024-04-25T10:54:30.728660+0000","last_clean":"2024-04-25T10:54:30.728660+0000","last_became_active":"2024-04-25T10:52:32.472113+0000","last_became_peered":"2024-04-25T10:52:32.472113+0000","last_unstale":"2024-04-25T10:54:30.728660+0000","last_undegraded":"2024-04-25T10:54:30.728660+0000","last_fullsized":"2024-04-25T10:54:30.728660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:20.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213703+0000","last_change":"2024-04-25T10:52:31.696488+0000","last_active":"2024-04-25T10:54:30.213703+0000","last_peered":"2024-04-25T10:54:30.213703+0000","last_clean":"2024-04-25T10:54:30.213703+0000","last_became_active":"2024-04-25T10:52:31.696062+0000","last_became_peered":"2024-04-25T10:52:31.696062+0000","last_unstale":"2024-04-25T10:54:30.213703+0000","last_undegraded":"2024-04-25T10:54:30.213703+0000","last_fullsized":"2024-04-25T10:54:30.213703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:24.329938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907510+0000","last_change":"2024-04-25T10:52:32.474555+0000","last_active":"2024-04-25T10:54:29.907510+0000","last_peered":"2024-04-25T10:54:29.907510+0000","last_clean":"2024-04-25T10:54:29.907510+0000","last_became_active":"2024-04-25T10:52:32.474001+0000","last_became_peered":"2024-04-25T10:52:32.474001+0000","last_unstale":"2024-04-25T10:54:29.907510+0000","last_undegraded":"2024-04-25T10:54:29.907510+0000","last_fullsized":"2024-04-25T10:54:29.907510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.775966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025259+0000","last_change":"2024-04-25T10:52:31.702251+0000","last_active":"2024-04-25T10:54:30.025259+0000","last_peered":"2024-04-25T10:54:30.025259+0000","last_clean":"2024-04-25T10:54:30.025259+0000","last_became_active":"2024-04-25T10:52:31.701763+0000","last_became_peered":"2024-04-25T10:52:31.701763+0000","last_unstale":"2024-04-25T10:54:30.025259+0000","last_undegraded":"2024-04-25T10:54:30.025259+0000","last_fullsized":"2024-04-25T10:54:30.025259+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954123+0000","last_change":"2024-04-25T10:52:32.465494+0000","last_active":"2024-04-25T10:54:29.954123+0000","last_peered":"2024-04-25T10:54:29.954123+0000","last_clean":"2024-04-25T10:54:29.954123+0000","last_became_active":"2024-04-25T10:52:32.463855+0000","last_became_peered":"2024-04-25T10:52:32.463855+0000","last_unstale":"2024-04-25T10:54:29.954123+0000","last_undegraded":"2024-04-25T10:54:29.954123+0000","last_fullsized":"2024-04-25T10:54:29.954123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:40.970580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213890+0000","last_change":"2024-04-25T10:52:31.700749+0000","last_active":"2024-04-25T10:54:30.213890+0000","last_peered":"2024-04-25T10:54:30.213890+0000","last_clean":"2024-04-25T10:54:30.213890+0000","last_became_active":"2024-04-25T10:52:31.699109+0000","last_became_peered":"2024-04-25T10:52:31.699109+0000","last_unstale":"2024-04-25T10:54:30.213890+0000","last_undegraded":"2024-04-25T10:54:30.213890+0000","last_fullsized":"2024-04-25T10:54:30.213890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:48.405333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954306+0000","last_change":"2024-04-25T10:52:32.468292+0000","last_active":"2024-04-25T10:54:29.954306+0000","last_peered":"2024-04-25T10:54:29.954306+0000","last_clean":"2024-04-25T10:54:29.954306+0000","last_became_active":"2024-04-25T10:52:32.467934+0000","last_became_peered":"2024-04-25T10:52:32.467934+0000","last_unstale":"2024-04-25T10:54:29.954306+0000","last_undegraded":"2024-04-25T10:54:29.954306+0000","last_fullsized":"2024-04-25T10:54:29.954306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.366735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.153767+0000","last_change":"2024-04-25T10:52:31.705025+0000","last_active":"2024-04-25T10:54:30.153767+0000","last_peered":"2024-04-25T10:54:30.153767+0000","last_clean":"2024-04-25T10:54:30.153767+0000","last_became_active":"2024-04-25T10:52:31.704532+0000","last_became_peered":"2024-04-25T10:52:31.704532+0000","last_unstale":"2024-04-25T10:54:30.153767+0000","last_undegraded":"2024-04-25T10:54:30.153767+0000","last_fullsized":"2024-04-25T10:54:30.153767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.371778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954100+0000","last_change":"2024-04-25T10:53:31.158758+0000","last_active":"2024-04-25T10:54:29.954100+0000","last_peered":"2024-04-25T10:54:29.954100+0000","last_clean":"2024-04-25T10:54:29.954100+0000","last_became_active":"2024-04-25T10:53:31.152575+0000","last_became_peered":"2024-04-25T10:53:31.152575+0000","last_unstale":"2024-04-25T10:54:29.954100+0000","last_undegraded":"2024-04-25T10:54:29.954100+0000","last_fullsized":"2024-04-25T10:54:29.954100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728882+0000","last_change":"2024-04-25T10:52:31.705770+0000","last_active":"2024-04-25T10:54:30.728882+0000","last_peered":"2024-04-25T10:54:30.728882+0000","last_clean":"2024-04-25T10:54:30.728882+0000","last_became_active":"2024-04-25T10:52:31.705391+0000","last_became_peered":"2024-04-25T10:52:31.705391+0000","last_unstale":"2024-04-25T10:54:30.728882+0000","last_undegraded":"2024-04-25T10:54:30.728882+0000","last_fullsized":"2024-04-25T10:54:30.728882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:32.831435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954387+0000","last_change":"2024-04-25T10:52:32.465550+0000","last_active":"2024-04-25T10:54:29.954387+0000","last_peered":"2024-04-25T10:54:29.954387+0000","last_clean":"2024-04-25T10:54:29.954387+0000","last_became_active":"2024-04-25T10:52:32.465034+0000","last_became_peered":"2024-04-25T10:52:32.465034+0000","last_unstale":"2024-04-25T10:54:29.954387+0000","last_undegraded":"2024-04-25T10:54:29.954387+0000","last_fullsized":"2024-04-25T10:54:29.954387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.810469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728670+0000","last_change":"2024-04-25T10:52:32.470812+0000","last_active":"2024-04-25T10:54:30.728670+0000","last_peered":"2024-04-25T10:54:30.728670+0000","last_clean":"2024-04-25T10:54:30.728670+0000","last_became_active":"2024-04-25T10:52:32.469620+0000","last_became_peered":"2024-04-25T10:52:32.469620+0000","last_unstale":"2024-04-25T10:54:30.728670+0000","last_undegraded":"2024-04-25T10:54:30.728670+0000","last_fullsized":"2024-04-25T10:54:30.728670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213694+0000","last_change":"2024-04-25T10:52:32.464762+0000","last_active":"2024-04-25T10:54:30.213694+0000","last_peered":"2024-04-25T10:54:30.213694+0000","last_clean":"2024-04-25T10:54:30.213694+0000","last_became_active":"2024-04-25T10:52:32.464316+0000","last_became_peered":"2024-04-25T10:52:32.464316+0000","last_unstale":"2024-04-25T10:54:30.213694+0000","last_undegraded":"2024-04-25T10:54:30.213694+0000","last_fullsized":"2024-04-25T10:54:30.213694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:55.526572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213902+0000","last_change":"2024-04-25T10:52:32.466429+0000","last_active":"2024-04-25T10:54:30.213902+0000","last_peered":"2024-04-25T10:54:30.213902+0000","last_clean":"2024-04-25T10:54:30.213902+0000","last_became_active":"2024-04-25T10:52:32.465132+0000","last_became_peered":"2024-04-25T10:52:32.465132+0000","last_unstale":"2024-04-25T10:54:30.213902+0000","last_undegraded":"2024-04-25T10:54:30.213902+0000","last_fullsized":"2024-04-25T10:54:30.213902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:14.274441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729722+0000","last_change":"2024-04-25T10:52:32.464666+0000","last_active":"2024-04-25T10:54:30.729722+0000","last_peered":"2024-04-25T10:54:30.729722+0000","last_clean":"2024-04-25T10:54:30.729722+0000","last_became_active":"2024-04-25T10:52:32.464226+0000","last_became_peered":"2024-04-25T10:52:32.464226+0000","last_unstale":"2024-04-25T10:54:30.729722+0000","last_undegraded":"2024-04-25T10:54:30.729722+0000","last_fullsized":"2024-04-25T10:54:30.729722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:39.388542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728837+0000","last_change":"2024-04-25T10:52:32.465199+0000","last_active":"2024-04-25T10:54:30.728837+0000","last_peered":"2024-04-25T10:54:30.728837+0000","last_clean":"2024-04-25T10:54:30.728837+0000","last_became_active":"2024-04-25T10:52:32.464285+0000","last_became_peered":"2024-04-25T10:52:32.464285+0000","last_unstale":"2024-04-25T10:54:30.728837+0000","last_undegraded":"2024-04-25T10:54:30.728837+0000","last_fullsized":"2024-04-25T10:54:30.728837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.010994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728913+0000","last_change":"2024-04-25T10:52:32.469140+0000","last_active":"2024-04-25T10:54:30.728913+0000","last_peered":"2024-04-25T10:54:30.728913+0000","last_clean":"2024-04-25T10:54:30.728913+0000","last_became_active":"2024-04-25T10:52:32.468344+0000","last_became_peered":"2024-04-25T10:52:32.468344+0000","last_unstale":"2024-04-25T10:54:30.728913+0000","last_undegraded":"2024-04-25T10:54:30.728913+0000","last_fullsized":"2024-04-25T10:54:30.728913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:07.228059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.214104+0000","last_change":"2024-04-25T10:52:32.469326+0000","last_active":"2024-04-25T10:54:30.214104+0000","last_peered":"2024-04-25T10:54:30.214104+0000","last_clean":"2024-04-25T10:54:30.214104+0000","last_became_active":"2024-04-25T10:52:32.468905+0000","last_became_peered":"2024-04-25T10:52:32.468905+0000","last_unstale":"2024-04-25T10:54:30.214104+0000","last_undegraded":"2024-04-25T10:54:30.214104+0000","last_fullsized":"2024-04-25T10:54:30.214104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:31.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907658+0000","last_change":"2024-04-25T10:52:32.459391+0000","last_active":"2024-04-25T10:54:29.907658+0000","last_peered":"2024-04-25T10:54:29.907658+0000","last_clean":"2024-04-25T10:54:29.907658+0000","last_became_active":"2024-04-25T10:52:32.458702+0000","last_became_peered":"2024-04-25T10:52:32.458702+0000","last_unstale":"2024-04-25T10:54:29.907658+0000","last_undegraded":"2024-04-25T10:54:29.907658+0000","last_fullsized":"2024-04-25T10:54:29.907658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.475714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908170+0000","last_change":"2024-04-25T10:52:32.467395+0000","last_active":"2024-04-25T10:54:29.908170+0000","last_peered":"2024-04-25T10:54:29.908170+0000","last_clean":"2024-04-25T10:54:29.908170+0000","last_became_active":"2024-04-25T10:52:32.466976+0000","last_became_peered":"2024-04-25T10:52:32.466976+0000","last_unstale":"2024-04-25T10:54:29.908170+0000","last_undegraded":"2024-04-25T10:54:29.908170+0000","last_fullsized":"2024-04-25T10:54:29.908170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:22.011064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907007+0000","last_change":"2024-04-25T10:52:32.458874+0000","last_active":"2024-04-25T10:54:29.907007+0000","last_peered":"2024-04-25T10:54:29.907007+0000","last_clean":"2024-04-25T10:54:29.907007+0000","last_became_active":"2024-04-25T10:52:32.458517+0000","last_became_peered":"2024-04-25T10:52:32.458517+0000","last_unstale":"2024-04-25T10:54:29.907007+0000","last_undegraded":"2024-04-25T10:54:29.907007+0000","last_fullsized":"2024-04-25T10:54:29.907007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:12.634521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213701+0000","last_change":"2024-04-25T10:52:32.473068+0000","last_active":"2024-04-25T10:54:30.213701+0000","last_peered":"2024-04-25T10:54:30.213701+0000","last_clean":"2024-04-25T10:54:30.213701+0000","last_became_active":"2024-04-25T10:52:32.472713+0000","last_became_peered":"2024-04-25T10:52:32.472713+0000","last_unstale":"2024-04-25T10:54:30.213701+0000","last_undegraded":"2024-04-25T10:54:30.213701+0000","last_fullsized":"2024-04-25T10:54:30.213701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:43.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729226+0000","last_change":"2024-04-25T10:52:32.468492+0000","last_active":"2024-04-25T10:54:30.729226+0000","last_peered":"2024-04-25T10:54:30.729226+0000","last_clean":"2024-04-25T10:54:30.729226+0000","last_became_active":"2024-04-25T10:52:32.468155+0000","last_became_peered":"2024-04-25T10:52:32.468155+0000","last_unstale":"2024-04-25T10:54:30.729226+0000","last_undegraded":"2024-04-25T10:54:30.729226+0000","last_fullsized":"2024-04-25T10:54:30.729226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.934230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728626+0000","last_change":"2024-04-25T10:52:32.463057+0000","last_active":"2024-04-25T10:54:30.728626+0000","last_peered":"2024-04-25T10:54:30.728626+0000","last_clean":"2024-04-25T10:54:30.728626+0000","last_became_active":"2024-04-25T10:52:32.460983+0000","last_became_peered":"2024-04-25T10:52:32.460983+0000","last_unstale":"2024-04-25T10:54:30.728626+0000","last_undegraded":"2024-04-25T10:54:30.728626+0000","last_fullsized":"2024-04-25T10:54:30.728626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593372,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":338,"num_read_kb":270,"num_write":197,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1780116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":411149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841577,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32820,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712332,"statfs":{"total":95995035648,"available":95961427968,"internally_reserved":0,"allocated":1683456,"data_stored":1210113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32412,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712740,"statfs":{"total":95995035648,"available":95961845760,"internally_reserved":0,"allocated":1261568,"data_stored":813991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28252,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716900,"statfs":{"total":95995035648,"available":95966105600,"internally_reserved":0,"allocated":1261568,"data_stored":813896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1462272,"data_stored":1011143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":847872,"data_stored":411266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":478125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":411239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":487868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":872448,"data_stored":420297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":411171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1142343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":199552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:54:31.348 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-25T10:54:31.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.498+0000 7f73389fe640 1 -- 172.21.15.19:0/2056876620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73341096d0 msgr2=0x7f733410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.498+0000 7f73389fe640 1 --2- 172.21.15.19:0/2056876620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73341096d0 0x7f733410baf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f732c009ca0 tx=0x7f732c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 -- 172.21.15.19:0/2056876620 shutdown_connections 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 --2- 172.21.15.19:0/2056876620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73341096d0 0x7f733410baf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 --2- 172.21.15.19:0/2056876620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7334106d70 0x7f7334109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 --2- 172.21.15.19:0/2056876620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7334104410 0x7f7334106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 -- 172.21.15.19:0/2056876620 >> 172.21.15.19:0/2056876620 conn(0x7f73340fda10 msgr2=0x7f73340ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 -- 172.21.15.19:0/2056876620 shutdown_connections 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 -- 172.21.15.19:0/2056876620 wait complete. 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.499+0000 7f73389fe640 1 Processor -- start 2024-04-25T10:54:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f73389fe640 1 -- start start 2024-04-25T10:54:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f73389fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7334104410 0x7f7334103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f73389fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334106d70 0x7f7334101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f73389fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73341096d0 0x7f7334102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f73389fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f733410f430 con 0x7f7334106d70 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f73389fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f733410f2b0 con 0x7f73341096d0 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f73389fe640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f733410f5b0 con 0x7f7334104410 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334106d70 0x7f7334101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334106d70 0x7f7334101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43514/0 (socket says 172.21.15.19:43514) 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 -- 172.21.15.19:0/1870982224 learned_addr learned my addr 172.21.15.19:0/1870982224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733359e640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73341096d0 0x7f7334102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f7332d9d640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7334104410 0x7f7334103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 -- 172.21.15.19:0/1870982224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7334104410 msgr2=0x7f7334103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7334104410 0x7f7334103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 -- 172.21.15.19:0/1870982224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73341096d0 msgr2=0x7f7334102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73341096d0 0x7f7334102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 -- 172.21.15.19:0/1870982224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334102b70 con 0x7f7334106d70 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f7332d9d640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7334104410 0x7f7334103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.500+0000 7f733259c640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334106d70 0x7f7334101f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f732800d950 tx=0x7f732800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.501+0000 7f731bfff640 1 -- 172.21.15.19:0/1870982224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328017030 con 0x7f7334106d70 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.501+0000 7f731bfff640 1 -- 172.21.15.19:0/1870982224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7328004020 con 0x7f7334106d70 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.501+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73341a40a0 con 0x7f7334106d70 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.501+0000 7f731bfff640 1 -- 172.21.15.19:0/1870982224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73280047a0 con 0x7f7334106d70 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.501+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73341a4560 con 0x7f7334106d70 2024-04-25T10:54:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.502+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73340ff530 con 0x7f7334106d70 2024-04-25T10:54:31.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.503+0000 7f731bfff640 1 -- 172.21.15.19:0/1870982224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f7328009a30 con 0x7f7334106d70 2024-04-25T10:54:31.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.504+0000 7f731bfff640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f731c0af8e0 0x7f731c0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.504+0000 7f731bfff640 1 -- 172.21.15.19:0/1870982224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f73280de3c0 con 0x7f7334106d70 2024-04-25T10:54:31.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.504+0000 7f7332d9d640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f731c0af8e0 0x7f731c0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.504+0000 7f7332d9d640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f731c0af8e0 0x7f731c0b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f73240058c0 tx=0x7f7324005ec0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:31.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.509+0000 7f731bfff640 1 -- 172.21.15.19:0/1870982224 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73280a7160 con 0x7f7334106d70 2024-04-25T10:54:31.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.694+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7334062fe0 con 0x7f731c0af8e0 2024-04-25T10:54:31.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.702+0000 7f731bfff640 1 -- 172.21.15.19:0/1870982224 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257649 (secure 0 0 0) 0x7f7334062fe0 con 0x7f731c0af8e0 2024-04-25T10:54:31.705 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:54:31.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f731c0af8e0 msgr2=0x7f731c0b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f731c0af8e0 0x7f731c0b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f73240058c0 tx=0x7f7324005ec0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334106d70 msgr2=0x7f7334101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334106d70 0x7f7334101f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f732800d950 tx=0x7f732800de10 comp rx=0 tx=0).stop 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 shutdown_connections 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f731c0af8e0 0x7f731c0b1df0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73341096d0 0x7f7334102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334106d70 0x7f7334101f00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 --2- 172.21.15.19:0/1870982224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7334104410 0x7f7334103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.705+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 >> 172.21.15.19:0/1870982224 conn(0x7f73340fda10 msgr2=0x7f7334104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.706+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 shutdown_connections 2024-04-25T10:54:31.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.706+0000 7f73389fe640 1 -- 172.21.15.19:0/1870982224 wait complete. 2024-04-25T10:54:31.717 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":3,"stamp":"2024-04-25T10:54:30.947976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183974,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":419,"num_read_kb":339,"num_write":322,"num_write_kb":3556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366416,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575408,"statfs":{"total":1151940427776,"available":1151565217792,"internally_reserved":0,"allocated":13398016,"data_stored":8022601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729039+0000","last_change":"2024-04-25T10:52:32.474738+0000","last_active":"2024-04-25T10:54:30.729039+0000","last_peered":"2024-04-25T10:54:30.729039+0000","last_clean":"2024-04-25T10:54:30.729039+0000","last_became_active":"2024-04-25T10:52:32.473812+0000","last_became_peered":"2024-04-25T10:52:32.473812+0000","last_unstale":"2024-04-25T10:54:30.729039+0000","last_undegraded":"2024-04-25T10:54:30.729039+0000","last_fullsized":"2024-04-25T10:54:30.729039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:25.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729032+0000","last_change":"2024-04-25T10:52:32.475491+0000","last_active":"2024-04-25T10:54:30.729032+0000","last_peered":"2024-04-25T10:54:30.729032+0000","last_clean":"2024-04-25T10:54:30.729032+0000","last_became_active":"2024-04-25T10:52:32.474164+0000","last_became_peered":"2024-04-25T10:52:32.474164+0000","last_unstale":"2024-04-25T10:54:30.729032+0000","last_undegraded":"2024-04-25T10:54:30.729032+0000","last_fullsized":"2024-04-25T10:54:30.729032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:58.838292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729884+0000","last_change":"2024-04-25T10:52:32.471349+0000","last_active":"2024-04-25T10:54:30.729884+0000","last_peered":"2024-04-25T10:54:30.729884+0000","last_clean":"2024-04-25T10:54:30.729884+0000","last_became_active":"2024-04-25T10:52:32.470972+0000","last_became_peered":"2024-04-25T10:52:32.470972+0000","last_unstale":"2024-04-25T10:54:30.729884+0000","last_undegraded":"2024-04-25T10:54:30.729884+0000","last_fullsized":"2024-04-25T10:54:30.729884+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:28.634537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728967+0000","last_change":"2024-04-25T10:52:32.470212+0000","last_active":"2024-04-25T10:54:30.728967+0000","last_peered":"2024-04-25T10:54:30.728967+0000","last_clean":"2024-04-25T10:54:30.728967+0000","last_became_active":"2024-04-25T10:52:32.469642+0000","last_became_peered":"2024-04-25T10:52:32.469642+0000","last_unstale":"2024-04-25T10:54:30.728967+0000","last_undegraded":"2024-04-25T10:54:30.728967+0000","last_fullsized":"2024-04-25T10:54:30.728967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:08.231679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025284+0000","last_change":"2024-04-25T10:52:32.456568+0000","last_active":"2024-04-25T10:54:30.025284+0000","last_peered":"2024-04-25T10:54:30.025284+0000","last_clean":"2024-04-25T10:54:30.025284+0000","last_became_active":"2024-04-25T10:52:32.456051+0000","last_became_peered":"2024-04-25T10:52:32.456051+0000","last_unstale":"2024-04-25T10:54:30.025284+0000","last_undegraded":"2024-04-25T10:54:30.025284+0000","last_fullsized":"2024-04-25T10:54:30.025284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908129+0000","last_change":"2024-04-25T10:52:32.479781+0000","last_active":"2024-04-25T10:54:29.908129+0000","last_peered":"2024-04-25T10:54:29.908129+0000","last_clean":"2024-04-25T10:54:29.908129+0000","last_became_active":"2024-04-25T10:52:32.479213+0000","last_became_peered":"2024-04-25T10:52:32.479213+0000","last_unstale":"2024-04-25T10:54:29.908129+0000","last_undegraded":"2024-04-25T10:54:29.908129+0000","last_fullsized":"2024-04-25T10:54:29.908129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:16.239969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953981+0000","last_change":"2024-04-25T10:52:32.471430+0000","last_active":"2024-04-25T10:54:29.953981+0000","last_peered":"2024-04-25T10:54:29.953981+0000","last_clean":"2024-04-25T10:54:29.953981+0000","last_became_active":"2024-04-25T10:52:32.470518+0000","last_became_peered":"2024-04-25T10:52:32.470518+0000","last_unstale":"2024-04-25T10:54:29.953981+0000","last_undegraded":"2024-04-25T10:54:29.953981+0000","last_fullsized":"2024-04-25T10:54:29.953981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:12.846174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.906963+0000","last_change":"2024-04-25T10:52:32.472545+0000","last_active":"2024-04-25T10:54:29.906963+0000","last_peered":"2024-04-25T10:54:29.906963+0000","last_clean":"2024-04-25T10:54:29.906963+0000","last_became_active":"2024-04-25T10:52:32.472151+0000","last_became_peered":"2024-04-25T10:52:32.472151+0000","last_unstale":"2024-04-25T10:54:29.906963+0000","last_undegraded":"2024-04-25T10:54:29.906963+0000","last_fullsized":"2024-04-25T10:54:29.906963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:36.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.214026+0000","last_change":"2024-04-25T10:52:32.466223+0000","last_active":"2024-04-25T10:54:30.214026+0000","last_peered":"2024-04-25T10:54:30.214026+0000","last_clean":"2024-04-25T10:54:30.214026+0000","last_became_active":"2024-04-25T10:52:32.465498+0000","last_became_peered":"2024-04-25T10:52:32.465498+0000","last_unstale":"2024-04-25T10:54:30.214026+0000","last_undegraded":"2024-04-25T10:54:30.214026+0000","last_fullsized":"2024-04-25T10:54:30.214026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:25.924450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908452+0000","last_change":"2024-04-25T10:52:32.470601+0000","last_active":"2024-04-25T10:54:29.908452+0000","last_peered":"2024-04-25T10:54:29.908452+0000","last_clean":"2024-04-25T10:54:29.908452+0000","last_became_active":"2024-04-25T10:52:32.468984+0000","last_became_peered":"2024-04-25T10:52:32.468984+0000","last_unstale":"2024-04-25T10:54:29.908452+0000","last_undegraded":"2024-04-25T10:54:29.908452+0000","last_fullsized":"2024-04-25T10:54:29.908452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954302+0000","last_change":"2024-04-25T10:52:32.463907+0000","last_active":"2024-04-25T10:54:29.954302+0000","last_peered":"2024-04-25T10:54:29.954302+0000","last_clean":"2024-04-25T10:54:29.954302+0000","last_became_active":"2024-04-25T10:52:32.463536+0000","last_became_peered":"2024-04-25T10:52:32.463536+0000","last_unstale":"2024-04-25T10:54:29.954302+0000","last_undegraded":"2024-04-25T10:54:29.954302+0000","last_fullsized":"2024-04-25T10:54:29.954302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.503805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728733+0000","last_change":"2024-04-25T10:52:32.473646+0000","last_active":"2024-04-25T10:54:30.728733+0000","last_peered":"2024-04-25T10:54:30.728733+0000","last_clean":"2024-04-25T10:54:30.728733+0000","last_became_active":"2024-04-25T10:52:32.473310+0000","last_became_peered":"2024-04-25T10:52:32.473310+0000","last_unstale":"2024-04-25T10:54:30.728733+0000","last_undegraded":"2024-04-25T10:54:30.728733+0000","last_fullsized":"2024-04-25T10:54:30.728733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.691293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728931+0000","last_change":"2024-04-25T10:53:31.244988+0000","last_active":"2024-04-25T10:54:30.728931+0000","last_peered":"2024-04-25T10:54:30.728931+0000","last_clean":"2024-04-25T10:54:30.728931+0000","last_became_active":"2024-04-25T10:53:31.244535+0000","last_became_peered":"2024-04-25T10:53:31.244535+0000","last_unstale":"2024-04-25T10:54:30.728931+0000","last_undegraded":"2024-04-25T10:54:30.728931+0000","last_fullsized":"2024-04-25T10:54:30.728931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:14.205691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729478+0000","last_change":"2024-04-25T10:52:32.476104+0000","last_active":"2024-04-25T10:54:30.729478+0000","last_peered":"2024-04-25T10:54:30.729478+0000","last_clean":"2024-04-25T10:54:30.729478+0000","last_became_active":"2024-04-25T10:52:32.475342+0000","last_became_peered":"2024-04-25T10:52:32.475342+0000","last_unstale":"2024-04-25T10:54:30.729478+0000","last_undegraded":"2024-04-25T10:54:30.729478+0000","last_fullsized":"2024-04-25T10:54:30.729478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:37.087793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729220+0000","last_change":"2024-04-25T10:52:32.470263+0000","last_active":"2024-04-25T10:54:30.729220+0000","last_peered":"2024-04-25T10:54:30.729220+0000","last_clean":"2024-04-25T10:54:30.729220+0000","last_became_active":"2024-04-25T10:52:32.469873+0000","last_became_peered":"2024-04-25T10:52:32.469873+0000","last_unstale":"2024-04-25T10:54:30.729220+0000","last_undegraded":"2024-04-25T10:54:30.729220+0000","last_fullsized":"2024-04-25T10:54:30.729220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.349598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213526+0000","last_change":"2024-04-25T10:52:32.471851+0000","last_active":"2024-04-25T10:54:30.213526+0000","last_peered":"2024-04-25T10:54:30.213526+0000","last_clean":"2024-04-25T10:54:30.213526+0000","last_became_active":"2024-04-25T10:52:32.471477+0000","last_became_peered":"2024-04-25T10:52:32.471477+0000","last_unstale":"2024-04-25T10:54:30.213526+0000","last_undegraded":"2024-04-25T10:54:30.213526+0000","last_fullsized":"2024-04-25T10:54:30.213526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:59.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728928+0000","last_change":"2024-04-25T10:52:32.472530+0000","last_active":"2024-04-25T10:54:30.728928+0000","last_peered":"2024-04-25T10:54:30.728928+0000","last_clean":"2024-04-25T10:54:30.728928+0000","last_became_active":"2024-04-25T10:52:32.472174+0000","last_became_peered":"2024-04-25T10:52:32.472174+0000","last_unstale":"2024-04-25T10:54:30.728928+0000","last_undegraded":"2024-04-25T10:54:30.728928+0000","last_fullsized":"2024-04-25T10:54:30.728928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:09.473575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022991+0000","last_change":"2024-04-25T10:52:32.473617+0000","last_active":"2024-04-25T10:54:30.022991+0000","last_peered":"2024-04-25T10:54:30.022991+0000","last_clean":"2024-04-25T10:54:30.022991+0000","last_became_active":"2024-04-25T10:52:32.473261+0000","last_became_peered":"2024-04-25T10:52:32.473261+0000","last_unstale":"2024-04-25T10:54:30.022991+0000","last_undegraded":"2024-04-25T10:54:30.022991+0000","last_fullsized":"2024-04-25T10:54:30.022991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:05.568672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213897+0000","last_change":"2024-04-25T10:52:32.466296+0000","last_active":"2024-04-25T10:54:30.213897+0000","last_peered":"2024-04-25T10:54:30.213897+0000","last_clean":"2024-04-25T10:54:30.213897+0000","last_became_active":"2024-04-25T10:52:32.463153+0000","last_became_peered":"2024-04-25T10:52:32.463153+0000","last_unstale":"2024-04-25T10:54:30.213897+0000","last_undegraded":"2024-04-25T10:54:30.213897+0000","last_fullsized":"2024-04-25T10:54:30.213897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.202795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022989+0000","last_change":"2024-04-25T10:52:31.701949+0000","last_active":"2024-04-25T10:54:30.022989+0000","last_peered":"2024-04-25T10:54:30.022989+0000","last_clean":"2024-04-25T10:54:30.022989+0000","last_became_active":"2024-04-25T10:52:31.701508+0000","last_became_peered":"2024-04-25T10:52:31.701508+0000","last_unstale":"2024-04-25T10:54:30.022989+0000","last_undegraded":"2024-04-25T10:54:30.022989+0000","last_fullsized":"2024-04-25T10:54:30.022989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:41.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907167+0000","last_change":"2024-04-25T10:52:32.468871+0000","last_active":"2024-04-25T10:54:29.907167+0000","last_peered":"2024-04-25T10:54:29.907167+0000","last_clean":"2024-04-25T10:54:29.907167+0000","last_became_active":"2024-04-25T10:52:32.468460+0000","last_became_peered":"2024-04-25T10:52:32.468460+0000","last_unstale":"2024-04-25T10:54:29.907167+0000","last_undegraded":"2024-04-25T10:54:29.907167+0000","last_fullsized":"2024-04-25T10:54:29.907167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:29.055639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907643+0000","last_change":"2024-04-25T10:52:31.705068+0000","last_active":"2024-04-25T10:54:29.907643+0000","last_peered":"2024-04-25T10:54:29.907643+0000","last_clean":"2024-04-25T10:54:29.907643+0000","last_became_active":"2024-04-25T10:52:31.704439+0000","last_became_peered":"2024-04-25T10:52:31.704439+0000","last_unstale":"2024-04-25T10:54:29.907643+0000","last_undegraded":"2024-04-25T10:54:29.907643+0000","last_fullsized":"2024-04-25T10:54:29.907643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.347653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729948+0000","last_change":"2024-04-25T10:52:32.476025+0000","last_active":"2024-04-25T10:54:30.729948+0000","last_peered":"2024-04-25T10:54:30.729948+0000","last_clean":"2024-04-25T10:54:30.729948+0000","last_became_active":"2024-04-25T10:52:32.475611+0000","last_became_peered":"2024-04-25T10:52:32.475611+0000","last_unstale":"2024-04-25T10:54:30.729948+0000","last_undegraded":"2024-04-25T10:54:30.729948+0000","last_fullsized":"2024-04-25T10:54:30.729948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:45.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025643+0000","last_change":"2024-04-25T10:52:31.703012+0000","last_active":"2024-04-25T10:54:30.025643+0000","last_peered":"2024-04-25T10:54:30.025643+0000","last_clean":"2024-04-25T10:54:30.025643+0000","last_became_active":"2024-04-25T10:52:31.702689+0000","last_became_peered":"2024-04-25T10:52:31.702689+0000","last_unstale":"2024-04-25T10:54:30.025643+0000","last_undegraded":"2024-04-25T10:54:30.025643+0000","last_fullsized":"2024-04-25T10:54:30.025643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:13.038859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213642+0000","last_change":"2024-04-25T10:52:32.475858+0000","last_active":"2024-04-25T10:54:30.213642+0000","last_peered":"2024-04-25T10:54:30.213642+0000","last_clean":"2024-04-25T10:54:30.213642+0000","last_became_active":"2024-04-25T10:52:32.475486+0000","last_became_peered":"2024-04-25T10:52:32.475486+0000","last_unstale":"2024-04-25T10:54:30.213642+0000","last_undegraded":"2024-04-25T10:54:30.213642+0000","last_fullsized":"2024-04-25T10:54:30.213642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.439208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728623+0000","last_change":"2024-04-25T10:53:31.155911+0000","last_active":"2024-04-25T10:54:30.728623+0000","last_peered":"2024-04-25T10:54:30.728623+0000","last_clean":"2024-04-25T10:54:30.728623+0000","last_became_active":"2024-04-25T10:53:31.151778+0000","last_became_peered":"2024-04-25T10:53:31.151778+0000","last_unstale":"2024-04-25T10:54:30.728623+0000","last_undegraded":"2024-04-25T10:54:30.728623+0000","last_fullsized":"2024-04-25T10:54:30.728623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:46.936583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908530+0000","last_change":"2024-04-25T10:53:31.152237+0000","last_active":"2024-04-25T10:54:29.908530+0000","last_peered":"2024-04-25T10:54:29.908530+0000","last_clean":"2024-04-25T10:54:29.908530+0000","last_became_active":"2024-04-25T10:53:31.151821+0000","last_became_peered":"2024-04-25T10:53:31.151821+0000","last_unstale":"2024-04-25T10:54:29.908530+0000","last_undegraded":"2024-04-25T10:54:29.908530+0000","last_fullsized":"2024-04-25T10:54:29.908530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:02.453018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908200+0000","last_change":"2024-04-25T10:52:31.699644+0000","last_active":"2024-04-25T10:54:29.908200+0000","last_peered":"2024-04-25T10:54:29.908200+0000","last_clean":"2024-04-25T10:54:29.908200+0000","last_became_active":"2024-04-25T10:52:31.699315+0000","last_became_peered":"2024-04-25T10:52:31.699315+0000","last_unstale":"2024-04-25T10:54:29.908200+0000","last_undegraded":"2024-04-25T10:54:29.908200+0000","last_fullsized":"2024-04-25T10:54:29.908200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:52.915096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025139+0000","last_change":"2024-04-25T10:52:32.463372+0000","last_active":"2024-04-25T10:54:30.025139+0000","last_peered":"2024-04-25T10:54:30.025139+0000","last_clean":"2024-04-25T10:54:30.025139+0000","last_became_active":"2024-04-25T10:52:32.462926+0000","last_became_peered":"2024-04-25T10:52:32.462926+0000","last_unstale":"2024-04-25T10:54:30.025139+0000","last_undegraded":"2024-04-25T10:54:30.025139+0000","last_fullsized":"2024-04-25T10:54:30.025139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:17.724268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908048+0000","last_change":"2024-04-25T10:52:31.695754+0000","last_active":"2024-04-25T10:54:29.908048+0000","last_peered":"2024-04-25T10:54:29.908048+0000","last_clean":"2024-04-25T10:54:29.908048+0000","last_became_active":"2024-04-25T10:52:31.695381+0000","last_became_peered":"2024-04-25T10:52:31.695381+0000","last_unstale":"2024-04-25T10:54:29.908048+0000","last_undegraded":"2024-04-25T10:54:29.908048+0000","last_fullsized":"2024-04-25T10:54:29.908048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:58.851534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908339+0000","last_change":"2024-04-25T10:53:31.154575+0000","last_active":"2024-04-25T10:54:29.908339+0000","last_peered":"2024-04-25T10:54:29.908339+0000","last_clean":"2024-04-25T10:54:29.908339+0000","last_became_active":"2024-04-25T10:53:31.154145+0000","last_became_peered":"2024-04-25T10:53:31.154145+0000","last_unstale":"2024-04-25T10:54:29.908339+0000","last_undegraded":"2024-04-25T10:54:29.908339+0000","last_fullsized":"2024-04-25T10:54:29.908339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:37.838587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728773+0000","last_change":"2024-04-25T10:52:31.695067+0000","last_active":"2024-04-25T10:54:30.728773+0000","last_peered":"2024-04-25T10:54:30.728773+0000","last_clean":"2024-04-25T10:54:30.728773+0000","last_became_active":"2024-04-25T10:52:31.694648+0000","last_became_peered":"2024-04-25T10:52:31.694648+0000","last_unstale":"2024-04-25T10:54:30.728773+0000","last_undegraded":"2024-04-25T10:54:30.728773+0000","last_fullsized":"2024-04-25T10:54:30.728773+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:09.654835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907056+0000","last_change":"2024-04-25T10:52:32.453501+0000","last_active":"2024-04-25T10:54:29.907056+0000","last_peered":"2024-04-25T10:54:29.907056+0000","last_clean":"2024-04-25T10:54:29.907056+0000","last_became_active":"2024-04-25T10:52:32.453113+0000","last_became_peered":"2024-04-25T10:52:32.453113+0000","last_unstale":"2024-04-25T10:54:29.907056+0000","last_undegraded":"2024-04-25T10:54:29.907056+0000","last_fullsized":"2024-04-25T10:54:29.907056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:07.340709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908088+0000","last_change":"2024-04-25T10:52:31.697946+0000","last_active":"2024-04-25T10:54:29.908088+0000","last_peered":"2024-04-25T10:54:29.908088+0000","last_clean":"2024-04-25T10:54:29.908088+0000","last_became_active":"2024-04-25T10:52:31.697612+0000","last_became_peered":"2024-04-25T10:52:31.697612+0000","last_unstale":"2024-04-25T10:54:29.908088+0000","last_undegraded":"2024-04-25T10:54:29.908088+0000","last_fullsized":"2024-04-25T10:54:29.908088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.280776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022464+0000","last_change":"2024-04-25T10:52:32.473537+0000","last_active":"2024-04-25T10:54:30.022464+0000","last_peered":"2024-04-25T10:54:30.022464+0000","last_clean":"2024-04-25T10:54:30.022464+0000","last_became_active":"2024-04-25T10:52:32.473164+0000","last_became_peered":"2024-04-25T10:52:32.473164+0000","last_unstale":"2024-04-25T10:54:30.022464+0000","last_undegraded":"2024-04-25T10:54:30.022464+0000","last_fullsized":"2024-04-25T10:54:30.022464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.008031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908381+0000","last_change":"2024-04-25T10:52:31.694207+0000","last_active":"2024-04-25T10:54:29.908381+0000","last_peered":"2024-04-25T10:54:29.908381+0000","last_clean":"2024-04-25T10:54:29.908381+0000","last_became_active":"2024-04-25T10:52:31.693777+0000","last_became_peered":"2024-04-25T10:52:31.693777+0000","last_unstale":"2024-04-25T10:54:29.908381+0000","last_undegraded":"2024-04-25T10:54:29.908381+0000","last_fullsized":"2024-04-25T10:54:29.908381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:43.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213960+0000","last_change":"2024-04-25T10:52:32.475545+0000","last_active":"2024-04-25T10:54:30.213960+0000","last_peered":"2024-04-25T10:54:30.213960+0000","last_clean":"2024-04-25T10:54:30.213960+0000","last_became_active":"2024-04-25T10:52:32.475195+0000","last_became_peered":"2024-04-25T10:52:32.475195+0000","last_unstale":"2024-04-25T10:54:30.213960+0000","last_undegraded":"2024-04-25T10:54:30.213960+0000","last_fullsized":"2024-04-25T10:54:30.213960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:49.900374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954240+0000","last_change":"2024-04-25T10:52:31.698184+0000","last_active":"2024-04-25T10:54:29.954240+0000","last_peered":"2024-04-25T10:54:29.954240+0000","last_clean":"2024-04-25T10:54:29.954240+0000","last_became_active":"2024-04-25T10:52:31.697816+0000","last_became_peered":"2024-04-25T10:52:31.697816+0000","last_unstale":"2024-04-25T10:54:29.954240+0000","last_undegraded":"2024-04-25T10:54:29.954240+0000","last_fullsized":"2024-04-25T10:54:29.954240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:44.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729953+0000","last_change":"2024-04-25T10:52:32.472437+0000","last_active":"2024-04-25T10:54:30.729953+0000","last_peered":"2024-04-25T10:54:30.729953+0000","last_clean":"2024-04-25T10:54:30.729953+0000","last_became_active":"2024-04-25T10:52:32.471944+0000","last_became_peered":"2024-04-25T10:52:32.471944+0000","last_unstale":"2024-04-25T10:54:30.729953+0000","last_undegraded":"2024-04-25T10:54:30.729953+0000","last_fullsized":"2024-04-25T10:54:30.729953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:11.047158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908612+0000","last_change":"2024-04-25T10:52:31.696943+0000","last_active":"2024-04-25T10:54:29.908612+0000","last_peered":"2024-04-25T10:54:29.908612+0000","last_clean":"2024-04-25T10:54:29.908612+0000","last_became_active":"2024-04-25T10:52:31.696561+0000","last_became_peered":"2024-04-25T10:52:31.696561+0000","last_unstale":"2024-04-25T10:54:29.908612+0000","last_undegraded":"2024-04-25T10:54:29.908612+0000","last_fullsized":"2024-04-25T10:54:29.908612+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:14.628800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213863+0000","last_change":"2024-04-25T10:52:32.461966+0000","last_active":"2024-04-25T10:54:30.213863+0000","last_peered":"2024-04-25T10:54:30.213863+0000","last_clean":"2024-04-25T10:54:30.213863+0000","last_became_active":"2024-04-25T10:52:32.461574+0000","last_became_peered":"2024-04-25T10:52:32.461574+0000","last_unstale":"2024-04-25T10:54:30.213863+0000","last_undegraded":"2024-04-25T10:54:30.213863+0000","last_fullsized":"2024-04-25T10:54:30.213863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.525599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":295,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213415+0000","last_change":"2024-04-25T10:52:31.690938+0000","last_active":"2024-04-25T10:54:30.213415+0000","last_peered":"2024-04-25T10:54:30.213415+0000","last_clean":"2024-04-25T10:54:30.213415+0000","last_became_active":"2024-04-25T10:52:31.690511+0000","last_became_peered":"2024-04-25T10:52:31.690511+0000","last_unstale":"2024-04-25T10:54:30.213415+0000","last_undegraded":"2024-04-25T10:54:30.213415+0000","last_fullsized":"2024-04-25T10:54:30.213415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:10.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":240,"num_read_kb":185,"num_write":34,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025452+0000","last_change":"2024-04-25T10:52:32.474871+0000","last_active":"2024-04-25T10:54:30.025452+0000","last_peered":"2024-04-25T10:54:30.025452+0000","last_clean":"2024-04-25T10:54:30.025452+0000","last_became_active":"2024-04-25T10:52:32.474463+0000","last_became_peered":"2024-04-25T10:52:32.474463+0000","last_unstale":"2024-04-25T10:54:30.025452+0000","last_undegraded":"2024-04-25T10:54:30.025452+0000","last_fullsized":"2024-04-25T10:54:30.025452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:36.719230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213752+0000","last_change":"2024-04-25T10:52:31.691006+0000","last_active":"2024-04-25T10:54:30.213752+0000","last_peered":"2024-04-25T10:54:30.213752+0000","last_clean":"2024-04-25T10:54:30.213752+0000","last_became_active":"2024-04-25T10:52:31.690611+0000","last_became_peered":"2024-04-25T10:52:31.690611+0000","last_unstale":"2024-04-25T10:54:30.213752+0000","last_undegraded":"2024-04-25T10:54:30.213752+0000","last_fullsized":"2024-04-25T10:54:30.213752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:37.781160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728620+0000","last_change":"2024-04-25T10:52:32.463331+0000","last_active":"2024-04-25T10:54:30.728620+0000","last_peered":"2024-04-25T10:54:30.728620+0000","last_clean":"2024-04-25T10:54:30.728620+0000","last_became_active":"2024-04-25T10:52:32.462579+0000","last_became_peered":"2024-04-25T10:52:32.462579+0000","last_unstale":"2024-04-25T10:54:30.728620+0000","last_undegraded":"2024-04-25T10:54:30.728620+0000","last_fullsized":"2024-04-25T10:54:30.728620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908256+0000","last_change":"2024-04-25T10:52:31.704675+0000","last_active":"2024-04-25T10:54:29.908256+0000","last_peered":"2024-04-25T10:54:29.908256+0000","last_clean":"2024-04-25T10:54:29.908256+0000","last_became_active":"2024-04-25T10:52:31.704236+0000","last_became_peered":"2024-04-25T10:52:31.704236+0000","last_unstale":"2024-04-25T10:54:29.908256+0000","last_undegraded":"2024-04-25T10:54:29.908256+0000","last_fullsized":"2024-04-25T10:54:29.908256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.637006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954166+0000","last_change":"2024-04-25T10:52:32.462960+0000","last_active":"2024-04-25T10:54:29.954166+0000","last_peered":"2024-04-25T10:54:29.954166+0000","last_clean":"2024-04-25T10:54:29.954166+0000","last_became_active":"2024-04-25T10:52:32.461993+0000","last_became_peered":"2024-04-25T10:52:32.461993+0000","last_unstale":"2024-04-25T10:54:29.954166+0000","last_undegraded":"2024-04-25T10:54:29.954166+0000","last_fullsized":"2024-04-25T10:54:29.954166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:01.803209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728552+0000","last_change":"2024-04-25T10:52:31.698206+0000","last_active":"2024-04-25T10:54:30.728552+0000","last_peered":"2024-04-25T10:54:30.728552+0000","last_clean":"2024-04-25T10:54:30.728552+0000","last_became_active":"2024-04-25T10:52:31.697814+0000","last_became_peered":"2024-04-25T10:52:31.697814+0000","last_unstale":"2024-04-25T10:54:30.728552+0000","last_undegraded":"2024-04-25T10:54:30.728552+0000","last_fullsized":"2024-04-25T10:54:30.728552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:02.849229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953813+0000","last_change":"2024-04-25T10:52:32.473551+0000","last_active":"2024-04-25T10:54:29.953813+0000","last_peered":"2024-04-25T10:54:29.953813+0000","last_clean":"2024-04-25T10:54:29.953813+0000","last_became_active":"2024-04-25T10:52:32.473227+0000","last_became_peered":"2024-04-25T10:52:32.473227+0000","last_unstale":"2024-04-25T10:54:29.953813+0000","last_undegraded":"2024-04-25T10:54:29.953813+0000","last_fullsized":"2024-04-25T10:54:29.953813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:37.742066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025114+0000","last_change":"2024-04-25T10:52:31.700670+0000","last_active":"2024-04-25T10:54:30.025114+0000","last_peered":"2024-04-25T10:54:30.025114+0000","last_clean":"2024-04-25T10:54:30.025114+0000","last_became_active":"2024-04-25T10:52:31.700234+0000","last_became_peered":"2024-04-25T10:52:31.700234+0000","last_unstale":"2024-04-25T10:54:30.025114+0000","last_undegraded":"2024-04-25T10:54:30.025114+0000","last_fullsized":"2024-04-25T10:54:30.025114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:21.795784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908220+0000","last_change":"2024-04-25T10:53:31.153193+0000","last_active":"2024-04-25T10:54:29.908220+0000","last_peered":"2024-04-25T10:54:29.908220+0000","last_clean":"2024-04-25T10:54:29.908220+0000","last_became_active":"2024-04-25T10:53:31.152723+0000","last_became_peered":"2024-04-25T10:53:31.152723+0000","last_unstale":"2024-04-25T10:54:29.908220+0000","last_undegraded":"2024-04-25T10:54:29.908220+0000","last_fullsized":"2024-04-25T10:54:29.908220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:42.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025555+0000","last_change":"2024-04-25T10:52:31.698507+0000","last_active":"2024-04-25T10:54:30.025555+0000","last_peered":"2024-04-25T10:54:30.025555+0000","last_clean":"2024-04-25T10:54:30.025555+0000","last_became_active":"2024-04-25T10:52:31.698148+0000","last_became_peered":"2024-04-25T10:52:31.698148+0000","last_unstale":"2024-04-25T10:54:30.025555+0000","last_undegraded":"2024-04-25T10:54:30.025555+0000","last_fullsized":"2024-04-25T10:54:30.025555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:48.993234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728798+0000","last_change":"2024-04-25T10:52:32.465874+0000","last_active":"2024-04-25T10:54:30.728798+0000","last_peered":"2024-04-25T10:54:30.728798+0000","last_clean":"2024-04-25T10:54:30.728798+0000","last_became_active":"2024-04-25T10:52:32.465511+0000","last_became_peered":"2024-04-25T10:52:32.465511+0000","last_unstale":"2024-04-25T10:54:30.728798+0000","last_undegraded":"2024-04-25T10:54:30.728798+0000","last_fullsized":"2024-04-25T10:54:30.728798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:15.274909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022791+0000","last_change":"2024-04-25T10:52:31.691074+0000","last_active":"2024-04-25T10:54:30.022791+0000","last_peered":"2024-04-25T10:54:30.022791+0000","last_clean":"2024-04-25T10:54:30.022791+0000","last_became_active":"2024-04-25T10:52:31.690402+0000","last_became_peered":"2024-04-25T10:52:31.690402+0000","last_unstale":"2024-04-25T10:54:30.022791+0000","last_undegraded":"2024-04-25T10:54:30.022791+0000","last_fullsized":"2024-04-25T10:54:30.022791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.529860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908342+0000","last_change":"2024-04-25T10:52:32.477968+0000","last_active":"2024-04-25T10:54:29.908342+0000","last_peered":"2024-04-25T10:54:29.908342+0000","last_clean":"2024-04-25T10:54:29.908342+0000","last_became_active":"2024-04-25T10:52:32.476395+0000","last_became_peered":"2024-04-25T10:52:32.476395+0000","last_unstale":"2024-04-25T10:54:29.908342+0000","last_undegraded":"2024-04-25T10:54:29.908342+0000","last_fullsized":"2024-04-25T10:54:29.908342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:13.128910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953969+0000","last_change":"2024-04-25T10:52:31.698934+0000","last_active":"2024-04-25T10:54:29.953969+0000","last_peered":"2024-04-25T10:54:29.953969+0000","last_clean":"2024-04-25T10:54:29.953969+0000","last_became_active":"2024-04-25T10:52:31.698565+0000","last_became_peered":"2024-04-25T10:52:31.698565+0000","last_unstale":"2024-04-25T10:54:29.953969+0000","last_undegraded":"2024-04-25T10:54:29.953969+0000","last_fullsized":"2024-04-25T10:54:29.953969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.360617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908305+0000","last_change":"2024-04-25T10:52:32.477538+0000","last_active":"2024-04-25T10:54:29.908305+0000","last_peered":"2024-04-25T10:54:29.908305+0000","last_clean":"2024-04-25T10:54:29.908305+0000","last_became_active":"2024-04-25T10:52:32.476088+0000","last_became_peered":"2024-04-25T10:52:32.476088+0000","last_unstale":"2024-04-25T10:54:29.908305+0000","last_undegraded":"2024-04-25T10:54:29.908305+0000","last_fullsized":"2024-04-25T10:54:29.908305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.987071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.022333+0000","last_change":"2024-04-25T10:53:31.244447+0000","last_active":"2024-04-25T10:54:30.022333+0000","last_peered":"2024-04-25T10:54:30.022333+0000","last_clean":"2024-04-25T10:54:30.022333+0000","last_became_active":"2024-04-25T10:53:31.243944+0000","last_became_peered":"2024-04-25T10:53:31.243944+0000","last_unstale":"2024-04-25T10:54:30.022333+0000","last_undegraded":"2024-04-25T10:54:30.022333+0000","last_fullsized":"2024-04-25T10:54:30.022333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:46.344278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728666+0000","last_change":"2024-04-25T10:52:31.706700+0000","last_active":"2024-04-25T10:54:30.728666+0000","last_peered":"2024-04-25T10:54:30.728666+0000","last_clean":"2024-04-25T10:54:30.728666+0000","last_became_active":"2024-04-25T10:52:31.706392+0000","last_became_peered":"2024-04-25T10:52:31.706392+0000","last_unstale":"2024-04-25T10:54:30.728666+0000","last_undegraded":"2024-04-25T10:54:30.728666+0000","last_fullsized":"2024-04-25T10:54:30.728666+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:58.867351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907365+0000","last_change":"2024-04-25T10:52:32.462603+0000","last_active":"2024-04-25T10:54:29.907365+0000","last_peered":"2024-04-25T10:54:29.907365+0000","last_clean":"2024-04-25T10:54:29.907365+0000","last_became_active":"2024-04-25T10:52:32.462238+0000","last_became_peered":"2024-04-25T10:52:32.462238+0000","last_unstale":"2024-04-25T10:54:29.907365+0000","last_undegraded":"2024-04-25T10:54:29.907365+0000","last_fullsized":"2024-04-25T10:54:29.907365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.555683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728675+0000","last_change":"2024-04-25T10:52:31.694054+0000","last_active":"2024-04-25T10:54:30.728675+0000","last_peered":"2024-04-25T10:54:30.728675+0000","last_clean":"2024-04-25T10:54:30.728675+0000","last_became_active":"2024-04-25T10:52:31.693693+0000","last_became_peered":"2024-04-25T10:52:31.693693+0000","last_unstale":"2024-04-25T10:54:30.728675+0000","last_undegraded":"2024-04-25T10:54:30.728675+0000","last_fullsized":"2024-04-25T10:54:30.728675+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:22.238131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.953885+0000","last_change":"2024-04-25T10:51:22.730925+0000","last_active":"2024-04-25T10:54:29.953885+0000","last_peered":"2024-04-25T10:54:29.953885+0000","last_clean":"2024-04-25T10:54:29.953885+0000","last_became_active":"2024-04-25T10:51:22.725168+0000","last_became_peered":"2024-04-25T10:51:22.725168+0000","last_unstale":"2024-04-25T10:54:29.953885+0000","last_undegraded":"2024-04-25T10:54:29.953885+0000","last_fullsized":"2024-04-25T10:54:29.953885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:47:56.758246+0000","last_clean_scrub_stamp":"2024-04-25T10:47:56.758246+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:02.416333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728494+0000","last_change":"2024-04-25T10:52:32.461608+0000","last_active":"2024-04-25T10:54:30.728494+0000","last_peered":"2024-04-25T10:54:30.728494+0000","last_clean":"2024-04-25T10:54:30.728494+0000","last_became_active":"2024-04-25T10:52:32.461210+0000","last_became_peered":"2024-04-25T10:52:32.461210+0000","last_unstale":"2024-04-25T10:54:30.728494+0000","last_undegraded":"2024-04-25T10:54:30.728494+0000","last_fullsized":"2024-04-25T10:54:30.728494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:14.897053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954025+0000","last_change":"2024-04-25T10:52:31.701003+0000","last_active":"2024-04-25T10:54:29.954025+0000","last_peered":"2024-04-25T10:54:29.954025+0000","last_clean":"2024-04-25T10:54:29.954025+0000","last_became_active":"2024-04-25T10:52:31.699623+0000","last_became_peered":"2024-04-25T10:52:31.699623+0000","last_unstale":"2024-04-25T10:54:29.954025+0000","last_undegraded":"2024-04-25T10:54:29.954025+0000","last_fullsized":"2024-04-25T10:54:29.954025+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:45.175760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025523+0000","last_change":"2024-04-25T10:52:31.702595+0000","last_active":"2024-04-25T10:54:30.025523+0000","last_peered":"2024-04-25T10:54:30.025523+0000","last_clean":"2024-04-25T10:54:30.025523+0000","last_became_active":"2024-04-25T10:52:31.702124+0000","last_became_peered":"2024-04-25T10:52:31.702124+0000","last_unstale":"2024-04-25T10:54:30.025523+0000","last_undegraded":"2024-04-25T10:54:30.025523+0000","last_fullsized":"2024-04-25T10:54:30.025523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:54.351859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728982+0000","last_change":"2024-04-25T10:52:32.467987+0000","last_active":"2024-04-25T10:54:30.728982+0000","last_peered":"2024-04-25T10:54:30.728982+0000","last_clean":"2024-04-25T10:54:30.728982+0000","last_became_active":"2024-04-25T10:52:32.467613+0000","last_became_peered":"2024-04-25T10:52:32.467613+0000","last_unstale":"2024-04-25T10:54:30.728982+0000","last_undegraded":"2024-04-25T10:54:30.728982+0000","last_fullsized":"2024-04-25T10:54:30.728982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:00.112102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'43","reported_seq":87,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.159606+0000","last_change":"2024-04-25T10:52:31.699868+0000","last_active":"2024-04-25T10:54:30.159606+0000","last_peered":"2024-04-25T10:54:30.159606+0000","last_clean":"2024-04-25T10:54:30.159606+0000","last_became_active":"2024-04-25T10:52:31.699470+0000","last_became_peered":"2024-04-25T10:52:31.699470+0000","last_unstale":"2024-04-25T10:54:30.159606+0000","last_undegraded":"2024-04-25T10:54:30.159606+0000","last_fullsized":"2024-04-25T10:54:30.159606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.161821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":77,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907687+0000","last_change":"2024-04-25T10:52:32.459082+0000","last_active":"2024-04-25T10:54:29.907687+0000","last_peered":"2024-04-25T10:54:29.907687+0000","last_clean":"2024-04-25T10:54:29.907687+0000","last_became_active":"2024-04-25T10:52:32.458190+0000","last_became_peered":"2024-04-25T10:52:32.458190+0000","last_unstale":"2024-04-25T10:54:29.907687+0000","last_undegraded":"2024-04-25T10:54:29.907687+0000","last_fullsized":"2024-04-25T10:54:29.907687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.280762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728730+0000","last_change":"2024-04-25T10:52:31.700636+0000","last_active":"2024-04-25T10:54:30.728730+0000","last_peered":"2024-04-25T10:54:30.728730+0000","last_clean":"2024-04-25T10:54:30.728730+0000","last_became_active":"2024-04-25T10:52:31.700268+0000","last_became_peered":"2024-04-25T10:52:31.700268+0000","last_unstale":"2024-04-25T10:54:30.728730+0000","last_undegraded":"2024-04-25T10:54:30.728730+0000","last_fullsized":"2024-04-25T10:54:30.728730+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.439491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729188+0000","last_change":"2024-04-25T10:52:32.467406+0000","last_active":"2024-04-25T10:54:30.729188+0000","last_peered":"2024-04-25T10:54:30.729188+0000","last_clean":"2024-04-25T10:54:30.729188+0000","last_became_active":"2024-04-25T10:52:32.467053+0000","last_became_peered":"2024-04-25T10:52:32.467053+0000","last_unstale":"2024-04-25T10:54:30.729188+0000","last_undegraded":"2024-04-25T10:54:30.729188+0000","last_fullsized":"2024-04-25T10:54:30.729188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213604+0000","last_change":"2024-04-25T10:53:31.301411+0000","last_active":"2024-04-25T10:54:30.213604+0000","last_peered":"2024-04-25T10:54:30.213604+0000","last_clean":"2024-04-25T10:54:30.213604+0000","last_became_active":"2024-04-25T10:53:31.301022+0000","last_became_peered":"2024-04-25T10:53:31.301022+0000","last_unstale":"2024-04-25T10:54:30.213604+0000","last_undegraded":"2024-04-25T10:54:30.213604+0000","last_fullsized":"2024-04-25T10:54:30.213604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:28.171444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025114+0000","last_change":"2024-04-25T10:52:32.472709+0000","last_active":"2024-04-25T10:54:30.025114+0000","last_peered":"2024-04-25T10:54:30.025114+0000","last_clean":"2024-04-25T10:54:30.025114+0000","last_became_active":"2024-04-25T10:52:32.472331+0000","last_became_peered":"2024-04-25T10:52:32.472331+0000","last_unstale":"2024-04-25T10:54:30.025114+0000","last_undegraded":"2024-04-25T10:54:30.025114+0000","last_fullsized":"2024-04-25T10:54:30.025114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.047682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907099+0000","last_change":"2024-04-25T10:52:31.697360+0000","last_active":"2024-04-25T10:54:29.907099+0000","last_peered":"2024-04-25T10:54:29.907099+0000","last_clean":"2024-04-25T10:54:29.907099+0000","last_became_active":"2024-04-25T10:52:31.696991+0000","last_became_peered":"2024-04-25T10:52:31.696991+0000","last_unstale":"2024-04-25T10:54:29.907099+0000","last_undegraded":"2024-04-25T10:54:29.907099+0000","last_fullsized":"2024-04-25T10:54:29.907099+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:38.395372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728660+0000","last_change":"2024-04-25T10:52:32.472505+0000","last_active":"2024-04-25T10:54:30.728660+0000","last_peered":"2024-04-25T10:54:30.728660+0000","last_clean":"2024-04-25T10:54:30.728660+0000","last_became_active":"2024-04-25T10:52:32.472113+0000","last_became_peered":"2024-04-25T10:52:32.472113+0000","last_unstale":"2024-04-25T10:54:30.728660+0000","last_undegraded":"2024-04-25T10:54:30.728660+0000","last_fullsized":"2024-04-25T10:54:30.728660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:20.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213703+0000","last_change":"2024-04-25T10:52:31.696488+0000","last_active":"2024-04-25T10:54:30.213703+0000","last_peered":"2024-04-25T10:54:30.213703+0000","last_clean":"2024-04-25T10:54:30.213703+0000","last_became_active":"2024-04-25T10:52:31.696062+0000","last_became_peered":"2024-04-25T10:52:31.696062+0000","last_unstale":"2024-04-25T10:54:30.213703+0000","last_undegraded":"2024-04-25T10:54:30.213703+0000","last_fullsized":"2024-04-25T10:54:30.213703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:24.329938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907510+0000","last_change":"2024-04-25T10:52:32.474555+0000","last_active":"2024-04-25T10:54:29.907510+0000","last_peered":"2024-04-25T10:54:29.907510+0000","last_clean":"2024-04-25T10:54:29.907510+0000","last_became_active":"2024-04-25T10:52:32.474001+0000","last_became_peered":"2024-04-25T10:52:32.474001+0000","last_unstale":"2024-04-25T10:54:29.907510+0000","last_undegraded":"2024-04-25T10:54:29.907510+0000","last_fullsized":"2024-04-25T10:54:29.907510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.775966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.025259+0000","last_change":"2024-04-25T10:52:31.702251+0000","last_active":"2024-04-25T10:54:30.025259+0000","last_peered":"2024-04-25T10:54:30.025259+0000","last_clean":"2024-04-25T10:54:30.025259+0000","last_became_active":"2024-04-25T10:52:31.701763+0000","last_became_peered":"2024-04-25T10:52:31.701763+0000","last_unstale":"2024-04-25T10:54:30.025259+0000","last_undegraded":"2024-04-25T10:54:30.025259+0000","last_fullsized":"2024-04-25T10:54:30.025259+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:29.639896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954123+0000","last_change":"2024-04-25T10:52:32.465494+0000","last_active":"2024-04-25T10:54:29.954123+0000","last_peered":"2024-04-25T10:54:29.954123+0000","last_clean":"2024-04-25T10:54:29.954123+0000","last_became_active":"2024-04-25T10:52:32.463855+0000","last_became_peered":"2024-04-25T10:52:32.463855+0000","last_unstale":"2024-04-25T10:54:29.954123+0000","last_undegraded":"2024-04-25T10:54:29.954123+0000","last_fullsized":"2024-04-25T10:54:29.954123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:40.970580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213890+0000","last_change":"2024-04-25T10:52:31.700749+0000","last_active":"2024-04-25T10:54:30.213890+0000","last_peered":"2024-04-25T10:54:30.213890+0000","last_clean":"2024-04-25T10:54:30.213890+0000","last_became_active":"2024-04-25T10:52:31.699109+0000","last_became_peered":"2024-04-25T10:52:31.699109+0000","last_unstale":"2024-04-25T10:54:30.213890+0000","last_undegraded":"2024-04-25T10:54:30.213890+0000","last_fullsized":"2024-04-25T10:54:30.213890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:48.405333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954306+0000","last_change":"2024-04-25T10:52:32.468292+0000","last_active":"2024-04-25T10:54:29.954306+0000","last_peered":"2024-04-25T10:54:29.954306+0000","last_clean":"2024-04-25T10:54:29.954306+0000","last_became_active":"2024-04-25T10:52:32.467934+0000","last_became_peered":"2024-04-25T10:52:32.467934+0000","last_unstale":"2024-04-25T10:54:29.954306+0000","last_undegraded":"2024-04-25T10:54:29.954306+0000","last_fullsized":"2024-04-25T10:54:29.954306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.366735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.153767+0000","last_change":"2024-04-25T10:52:31.705025+0000","last_active":"2024-04-25T10:54:30.153767+0000","last_peered":"2024-04-25T10:54:30.153767+0000","last_clean":"2024-04-25T10:54:30.153767+0000","last_became_active":"2024-04-25T10:52:31.704532+0000","last_became_peered":"2024-04-25T10:52:31.704532+0000","last_unstale":"2024-04-25T10:54:30.153767+0000","last_undegraded":"2024-04-25T10:54:30.153767+0000","last_fullsized":"2024-04-25T10:54:30.153767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.371778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954100+0000","last_change":"2024-04-25T10:53:31.158758+0000","last_active":"2024-04-25T10:54:29.954100+0000","last_peered":"2024-04-25T10:54:29.954100+0000","last_clean":"2024-04-25T10:54:29.954100+0000","last_became_active":"2024-04-25T10:53:31.152575+0000","last_became_peered":"2024-04-25T10:53:31.152575+0000","last_unstale":"2024-04-25T10:54:29.954100+0000","last_undegraded":"2024-04-25T10:54:29.954100+0000","last_fullsized":"2024-04-25T10:54:29.954100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728882+0000","last_change":"2024-04-25T10:52:31.705770+0000","last_active":"2024-04-25T10:54:30.728882+0000","last_peered":"2024-04-25T10:54:30.728882+0000","last_clean":"2024-04-25T10:54:30.728882+0000","last_became_active":"2024-04-25T10:52:31.705391+0000","last_became_peered":"2024-04-25T10:52:31.705391+0000","last_unstale":"2024-04-25T10:54:30.728882+0000","last_undegraded":"2024-04-25T10:54:30.728882+0000","last_fullsized":"2024-04-25T10:54:30.728882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:30.669853+0000","last_clean_scrub_stamp":"2024-04-25T10:52:30.669853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:32.831435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.954387+0000","last_change":"2024-04-25T10:52:32.465550+0000","last_active":"2024-04-25T10:54:29.954387+0000","last_peered":"2024-04-25T10:54:29.954387+0000","last_clean":"2024-04-25T10:54:29.954387+0000","last_became_active":"2024-04-25T10:52:32.465034+0000","last_became_peered":"2024-04-25T10:52:32.465034+0000","last_unstale":"2024-04-25T10:54:29.954387+0000","last_undegraded":"2024-04-25T10:54:29.954387+0000","last_fullsized":"2024-04-25T10:54:29.954387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.810469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728670+0000","last_change":"2024-04-25T10:52:32.470812+0000","last_active":"2024-04-25T10:54:30.728670+0000","last_peered":"2024-04-25T10:54:30.728670+0000","last_clean":"2024-04-25T10:54:30.728670+0000","last_became_active":"2024-04-25T10:52:32.469620+0000","last_became_peered":"2024-04-25T10:52:32.469620+0000","last_unstale":"2024-04-25T10:54:30.728670+0000","last_undegraded":"2024-04-25T10:54:30.728670+0000","last_fullsized":"2024-04-25T10:54:30.728670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213694+0000","last_change":"2024-04-25T10:52:32.464762+0000","last_active":"2024-04-25T10:54:30.213694+0000","last_peered":"2024-04-25T10:54:30.213694+0000","last_clean":"2024-04-25T10:54:30.213694+0000","last_became_active":"2024-04-25T10:52:32.464316+0000","last_became_peered":"2024-04-25T10:52:32.464316+0000","last_unstale":"2024-04-25T10:54:30.213694+0000","last_undegraded":"2024-04-25T10:54:30.213694+0000","last_fullsized":"2024-04-25T10:54:30.213694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:55.526572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213902+0000","last_change":"2024-04-25T10:52:32.466429+0000","last_active":"2024-04-25T10:54:30.213902+0000","last_peered":"2024-04-25T10:54:30.213902+0000","last_clean":"2024-04-25T10:54:30.213902+0000","last_became_active":"2024-04-25T10:52:32.465132+0000","last_became_peered":"2024-04-25T10:52:32.465132+0000","last_unstale":"2024-04-25T10:54:30.213902+0000","last_undegraded":"2024-04-25T10:54:30.213902+0000","last_fullsized":"2024-04-25T10:54:30.213902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:14.274441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729722+0000","last_change":"2024-04-25T10:52:32.464666+0000","last_active":"2024-04-25T10:54:30.729722+0000","last_peered":"2024-04-25T10:54:30.729722+0000","last_clean":"2024-04-25T10:54:30.729722+0000","last_became_active":"2024-04-25T10:52:32.464226+0000","last_became_peered":"2024-04-25T10:52:32.464226+0000","last_unstale":"2024-04-25T10:54:30.729722+0000","last_undegraded":"2024-04-25T10:54:30.729722+0000","last_fullsized":"2024-04-25T10:54:30.729722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:39.388542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728837+0000","last_change":"2024-04-25T10:52:32.465199+0000","last_active":"2024-04-25T10:54:30.728837+0000","last_peered":"2024-04-25T10:54:30.728837+0000","last_clean":"2024-04-25T10:54:30.728837+0000","last_became_active":"2024-04-25T10:52:32.464285+0000","last_became_peered":"2024-04-25T10:52:32.464285+0000","last_unstale":"2024-04-25T10:54:30.728837+0000","last_undegraded":"2024-04-25T10:54:30.728837+0000","last_fullsized":"2024-04-25T10:54:30.728837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.010994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728913+0000","last_change":"2024-04-25T10:52:32.469140+0000","last_active":"2024-04-25T10:54:30.728913+0000","last_peered":"2024-04-25T10:54:30.728913+0000","last_clean":"2024-04-25T10:54:30.728913+0000","last_became_active":"2024-04-25T10:52:32.468344+0000","last_became_peered":"2024-04-25T10:52:32.468344+0000","last_unstale":"2024-04-25T10:54:30.728913+0000","last_undegraded":"2024-04-25T10:54:30.728913+0000","last_fullsized":"2024-04-25T10:54:30.728913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:07.228059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.214104+0000","last_change":"2024-04-25T10:52:32.469326+0000","last_active":"2024-04-25T10:54:30.214104+0000","last_peered":"2024-04-25T10:54:30.214104+0000","last_clean":"2024-04-25T10:54:30.214104+0000","last_became_active":"2024-04-25T10:52:32.468905+0000","last_became_peered":"2024-04-25T10:52:32.468905+0000","last_unstale":"2024-04-25T10:54:30.214104+0000","last_undegraded":"2024-04-25T10:54:30.214104+0000","last_fullsized":"2024-04-25T10:54:30.214104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:31.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907658+0000","last_change":"2024-04-25T10:52:32.459391+0000","last_active":"2024-04-25T10:54:29.907658+0000","last_peered":"2024-04-25T10:54:29.907658+0000","last_clean":"2024-04-25T10:54:29.907658+0000","last_became_active":"2024-04-25T10:52:32.458702+0000","last_became_peered":"2024-04-25T10:52:32.458702+0000","last_unstale":"2024-04-25T10:54:29.907658+0000","last_undegraded":"2024-04-25T10:54:29.907658+0000","last_fullsized":"2024-04-25T10:54:29.907658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.475714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.908170+0000","last_change":"2024-04-25T10:52:32.467395+0000","last_active":"2024-04-25T10:54:29.908170+0000","last_peered":"2024-04-25T10:54:29.908170+0000","last_clean":"2024-04-25T10:54:29.908170+0000","last_became_active":"2024-04-25T10:52:32.466976+0000","last_became_peered":"2024-04-25T10:52:32.466976+0000","last_unstale":"2024-04-25T10:54:29.908170+0000","last_undegraded":"2024-04-25T10:54:29.908170+0000","last_fullsized":"2024-04-25T10:54:29.908170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:22.011064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:29.907007+0000","last_change":"2024-04-25T10:52:32.458874+0000","last_active":"2024-04-25T10:54:29.907007+0000","last_peered":"2024-04-25T10:54:29.907007+0000","last_clean":"2024-04-25T10:54:29.907007+0000","last_became_active":"2024-04-25T10:52:32.458517+0000","last_became_peered":"2024-04-25T10:52:32.458517+0000","last_unstale":"2024-04-25T10:54:29.907007+0000","last_undegraded":"2024-04-25T10:54:29.907007+0000","last_fullsized":"2024-04-25T10:54:29.907007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:12.634521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.213701+0000","last_change":"2024-04-25T10:52:32.473068+0000","last_active":"2024-04-25T10:54:30.213701+0000","last_peered":"2024-04-25T10:54:30.213701+0000","last_clean":"2024-04-25T10:54:30.213701+0000","last_became_active":"2024-04-25T10:52:32.472713+0000","last_became_peered":"2024-04-25T10:52:32.472713+0000","last_unstale":"2024-04-25T10:54:30.213701+0000","last_undegraded":"2024-04-25T10:54:30.213701+0000","last_fullsized":"2024-04-25T10:54:30.213701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:43.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.729226+0000","last_change":"2024-04-25T10:52:32.468492+0000","last_active":"2024-04-25T10:54:30.729226+0000","last_peered":"2024-04-25T10:54:30.729226+0000","last_clean":"2024-04-25T10:54:30.729226+0000","last_became_active":"2024-04-25T10:52:32.468155+0000","last_became_peered":"2024-04-25T10:52:32.468155+0000","last_unstale":"2024-04-25T10:54:30.729226+0000","last_undegraded":"2024-04-25T10:54:30.729226+0000","last_fullsized":"2024-04-25T10:54:30.729226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.934230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:54:30.728626+0000","last_change":"2024-04-25T10:52:32.463057+0000","last_active":"2024-04-25T10:54:30.728626+0000","last_peered":"2024-04-25T10:54:30.728626+0000","last_clean":"2024-04-25T10:54:30.728626+0000","last_became_active":"2024-04-25T10:52:32.460983+0000","last_became_peered":"2024-04-25T10:52:32.460983+0000","last_unstale":"2024-04-25T10:54:30.728626+0000","last_undegraded":"2024-04-25T10:54:30.728626+0000","last_fullsized":"2024-04-25T10:54:30.728626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:52:31.677387+0000","last_clean_scrub_stamp":"2024-04-25T10:52:31.677387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593372,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":338,"num_read_kb":270,"num_write":197,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1780116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":411149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841577,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32820,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712332,"statfs":{"total":95995035648,"available":95961427968,"internally_reserved":0,"allocated":1683456,"data_stored":1210113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32412,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712740,"statfs":{"total":95995035648,"available":95961845760,"internally_reserved":0,"allocated":1261568,"data_stored":813991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28252,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716900,"statfs":{"total":95995035648,"available":95966105600,"internally_reserved":0,"allocated":1261568,"data_stored":813896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1462272,"data_stored":1011143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":847872,"data_stored":411266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":478125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":411239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":487868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":872448,"data_stored":420297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":411171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1142343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":199552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:54:31.721 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 -- 172.21.15.19:0/292545247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340103b10 msgr2=0x7f0340104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 --2- 172.21.15.19:0/292545247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340103b10 0x7f0340104810 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0330009900 tx=0x7f033002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 -- 172.21.15.19:0/292545247 shutdown_connections 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 --2- 172.21.15.19:0/292545247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0340109a50 0x7f034010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 --2- 172.21.15.19:0/292545247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0340104d50 0x7f0340105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 --2- 172.21.15.19:0/292545247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340103b10 0x7f0340104810 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 -- 172.21.15.19:0/292545247 >> 172.21.15.19:0/292545247 conn(0x7f03400fed40 msgr2=0x7f03401011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 -- 172.21.15.19:0/292545247 shutdown_connections 2024-04-25T10:54:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 -- 172.21.15.19:0/292545247 wait complete. 2024-04-25T10:54:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.870+0000 7f0346092640 1 Processor -- start 2024-04-25T10:54:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f0346092640 1 -- start start 2024-04-25T10:54:31.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f0346092640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0340103b10 0x7f0340077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f0346092640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340104d50 0x7f0340076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f0346092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0340109a50 0x7f0340076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f0346092640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f034010f610 con 0x7f0340104d50 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f0346092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f034010f490 con 0x7f0340109a50 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f0346092640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f034010f790 con 0x7f0340103b10 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340104d50 0x7f0340076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0340109a50 0x7f0340076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340104d50 0x7f0340076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43524/0 (socket says 172.21.15.19:43524) 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033effd640 1 -- 172.21.15.19:0/225163531 learned_addr learned my addr 172.21.15.19:0/225163531 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033f7fe640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0340103b10 0x7f0340077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033ffff640 1 -- 172.21.15.19:0/225163531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0340103b10 msgr2=0x7f0340077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033ffff640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0340103b10 0x7f0340077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033ffff640 1 -- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340104d50 msgr2=0x7f0340076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033ffff640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340104d50 0x7f0340076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.871+0000 7f033ffff640 1 -- 172.21.15.19:0/225163531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0340076c70 con 0x7f0340109a50 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.872+0000 7f033ffff640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0340109a50 0x7f0340076540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0338009fd0 tx=0x7f033800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.872+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033800ce20 con 0x7f0340109a50 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.872+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0338005020 con 0x7f0340109a50 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.872+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338010690 con 0x7f0340109a50 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.872+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03401a4310 con 0x7f0340109a50 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.872+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03401a4780 con 0x7f0340109a50 2024-04-25T10:54:31.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.873+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0340108860 con 0x7f0340109a50 2024-04-25T10:54:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.874+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f03380054d0 con 0x7f0340109a50 2024-04-25T10:54:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.875+0000 7f033cff9640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f03280af8e0 0x7f03280b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.875+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f03380def80 con 0x7f0340109a50 2024-04-25T10:54:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.875+0000 7f033f7fe640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f03280af8e0 0x7f03280b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.876+0000 7f033f7fe640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f03280af8e0 0x7f03280b1df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0330005ab0 tx=0x7f0330031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:31.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.879+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03380a7d60 con 0x7f0340109a50 2024-04-25T10:54:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:31.929+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f03380a38f0 con 0x7f0340109a50 2024-04-25T10:54:31.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:31 smithi019 ceph-mon[29209]: mgrmap e22: x(active, since 1.0228s), standbys: z, y 2024-04-25T10:54:31.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:31 smithi019 ceph-mon[29209]: [25/Apr/2024:10:54:31] ENGINE Bus STARTING 2024-04-25T10:54:31.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:31 smithi019 ceph-mon[29209]: [25/Apr/2024:10:54:31] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T10:54:31.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:31 smithi019 ceph-mon[29209]: [25/Apr/2024:10:54:31] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T10:54:31.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:31 smithi019 ceph-mon[29209]: [25/Apr/2024:10:54:31] ENGINE Bus STARTED 2024-04-25T10:54:31.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:31 smithi019 ceph-mon[29209]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:54:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.119+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03400637d0 con 0x7f0340109a50 2024-04-25T10:54:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.121+0000 7f033cff9640 1 -- 172.21.15.19:0/225163531 <== mon.2 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23035 (secure 0 0 0) 0x7f03380d5e20 con 0x7f0340109a50 2024-04-25T10:54:32.123 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:32.123 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:54:29.901359+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/48031724":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/2535542967":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/4264937365":"2024-04-26T10:54:29.901303+0000","172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3805408646":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6800/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:54:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f03280af8e0 msgr2=0x7f03280b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f03280af8e0 0x7f03280b1df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0330005ab0 tx=0x7f0330031400 comp rx=0 tx=0).stop 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0340109a50 msgr2=0x7f0340076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0340109a50 0x7f0340076540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0338009fd0 tx=0x7f033800ef10 comp rx=0 tx=0).stop 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 shutdown_connections 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f03280af8e0 0x7f03280b1df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0340109a50 0x7f0340076540 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0340104d50 0x7f0340076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 --2- 172.21.15.19:0/225163531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0340103b10 0x7f0340077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 >> 172.21.15.19:0/225163531 conn(0x7f03400fed40 msgr2=0x7f0340105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 shutdown_connections 2024-04-25T10:54:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.125+0000 7f0346092640 1 -- 172.21.15.19:0/225163531 wait complete. 2024-04-25T10:54:32.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:54:32.139 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:54:32.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:31 smithi169 ceph-mon[33049]: mgrmap e22: x(active, since 1.0228s), standbys: z, y 2024-04-25T10:54:32.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:31 smithi169 ceph-mon[33049]: [25/Apr/2024:10:54:31] ENGINE Bus STARTING 2024-04-25T10:54:32.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:31 smithi169 ceph-mon[33049]: [25/Apr/2024:10:54:31] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T10:54:32.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:31 smithi169 ceph-mon[33049]: [25/Apr/2024:10:54:31] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T10:54:32.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:31 smithi169 ceph-mon[33049]: [25/Apr/2024:10:54:31] ENGINE Bus STARTED 2024-04-25T10:54:32.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:31 smithi169 ceph-mon[33049]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:54:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 -- 172.21.15.19:0/686895778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 msgr2=0x7f4528106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 --2- 172.21.15.19:0/686895778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f4528106690 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4518009ca0 tx=0x7f451802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 -- 172.21.15.19:0/686895778 shutdown_connections 2024-04-25T10:54:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 --2- 172.21.15.19:0/686895778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528109770 0x7f452810bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 --2- 172.21.15.19:0/686895778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528106c60 0x7f45281090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 --2- 172.21.15.19:0/686895778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f4528106690 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 -- 172.21.15.19:0/686895778 >> 172.21.15.19:0/686895778 conn(0x7f45280fdbd0 msgr2=0x7f4528100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 -- 172.21.15.19:0/686895778 shutdown_connections 2024-04-25T10:54:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.262+0000 7f452d867640 1 -- 172.21.15.19:0/686895778 wait complete. 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.263+0000 7f452d867640 1 Processor -- start 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.263+0000 7f452d867640 1 -- start start 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f452d867640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f452819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f452d867640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528106c60 0x7f452819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f452d867640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528109770 0x7f45281a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f452d867640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f452810f5e0 con 0x7f4528104270 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f452d867640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f452810f460 con 0x7f4528109770 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f452d867640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f452810f760 con 0x7f4528106c60 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f452819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f452819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43572/0 (socket says 172.21.15.19:43572) 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 -- 172.21.15.19:0/153810321 learned_addr learned my addr 172.21.15.19:0/153810321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f45267fc640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528106c60 0x7f452819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f45277fe640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528109770 0x7f45281a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 -- 172.21.15.19:0/153810321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528106c60 msgr2=0x7f452819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528106c60 0x7f452819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 -- 172.21.15.19:0/153810321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528109770 msgr2=0x7f45281a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528109770 0x7f45281a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.264+0000 7f4526ffd640 1 -- 172.21.15.19:0/153810321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45281a20b0 con 0x7f4528104270 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.265+0000 7f4526ffd640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f452819a570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f451802ade0 tx=0x7f45180317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.265+0000 7f452c865640 1 -- 172.21.15.19:0/153810321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518003c30 con 0x7f4528104270 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.265+0000 7f452c865640 1 -- 172.21.15.19:0/153810321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518003dd0 con 0x7f4528104270 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.265+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45281a2340 con 0x7f4528104270 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.265+0000 7f452c865640 1 -- 172.21.15.19:0/153810321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451802fb60 con 0x7f4528104270 2024-04-25T10:54:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.265+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45281a2890 con 0x7f4528104270 2024-04-25T10:54:32.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.266+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452806c2a0 con 0x7f4528104270 2024-04-25T10:54:32.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.267+0000 7f452c865640 1 -- 172.21.15.19:0/153810321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f451802fd00 con 0x7f4528104270 2024-04-25T10:54:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.268+0000 7f452c865640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f45100afc60 0x7f45100b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.268+0000 7f452c865640 1 -- 172.21.15.19:0/153810321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f4518100600 con 0x7f4528104270 2024-04-25T10:54:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.269+0000 7f45267fc640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f45100afc60 0x7f45100b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.269+0000 7f45267fc640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f45100afc60 0x7f45100b2170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f452819b610 tx=0x7f451c005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:32.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.273+0000 7f452c865640 1 -- 172.21.15.19:0/153810321 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45180c8f80 con 0x7f4528104270 2024-04-25T10:54:32.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:31 smithi112 ceph-mon[32799]: mgrmap e22: x(active, since 1.0228s), standbys: z, y 2024-04-25T10:54:32.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:31 smithi112 ceph-mon[32799]: [25/Apr/2024:10:54:31] ENGINE Bus STARTING 2024-04-25T10:54:32.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:31 smithi112 ceph-mon[32799]: [25/Apr/2024:10:54:31] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T10:54:32.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:31 smithi112 ceph-mon[32799]: [25/Apr/2024:10:54:31] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T10:54:32.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:31 smithi112 ceph-mon[32799]: [25/Apr/2024:10:54:31] ENGINE Bus STARTED 2024-04-25T10:54:32.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:31 smithi112 ceph-mon[32799]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:54:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.496+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45281a2af0 con 0x7f4528104270 2024-04-25T10:54:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.496+0000 7f452c865640 1 -- 172.21.15.19:0/153810321 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f45180c8880 con 0x7f4528104270 2024-04-25T10:54:32.497 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:54:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f45100afc60 msgr2=0x7f45100b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f45100afc60 0x7f45100b2170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f452819b610 tx=0x7f451c005f90 comp rx=0 tx=0).stop 2024-04-25T10:54:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 msgr2=0x7f452819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f452819a570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f451802ade0 tx=0x7f45180317c0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 shutdown_connections 2024-04-25T10:54:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f45100afc60 0x7f45100b2170 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528109770 0x7f45281a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528106c60 0x7f452819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 --2- 172.21.15.19:0/153810321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4528104270 0x7f452819a570 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 >> 172.21.15.19:0/153810321 conn(0x7f45280fdbd0 msgr2=0x7f45281048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 shutdown_connections 2024-04-25T10:54:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.499+0000 7f452d867640 1 -- 172.21.15.19:0/153810321 wait complete. 2024-04-25T10:54:32.511 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 -- 172.21.15.19:0/2498790551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 msgr2=0x7f97b010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 --2- 172.21.15.19:0/2498790551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b010be60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009ca0 tx=0x7f97a802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 -- 172.21.15.19:0/2498790551 shutdown_connections 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 --2- 172.21.15.19:0/2498790551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b010be60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 --2- 172.21.15.19:0/2498790551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0104d40 0x7f97b0105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 --2- 172.21.15.19:0/2498790551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b0103b00 0x7f97b0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 -- 172.21.15.19:0/2498790551 >> 172.21.15.19:0/2498790551 conn(0x7f97b00fed10 msgr2=0x7f97b01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 -- 172.21.15.19:0/2498790551 shutdown_connections 2024-04-25T10:54:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 -- 172.21.15.19:0/2498790551 wait complete. 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.631+0000 7f97b8261640 1 Processor -- start 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b8261640 1 -- start start 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b8261640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b0103b00 0x7f97b019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b8261640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0104d40 0x7f97b019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b8261640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b8261640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010f630 con 0x7f97b0109a40 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b8261640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010f4b0 con 0x7f97b0104d40 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b8261640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010f7b0 con 0x7f97b0103b00 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b01a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43596/0 (socket says 172.21.15.19:43596) 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 -- 172.21.15.19:0/685038147 learned_addr learned my addr 172.21.15.19:0/685038147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b57fc640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0104d40 0x7f97b019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b5ffd640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b0103b00 0x7f97b019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 -- 172.21.15.19:0/685038147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b0103b00 msgr2=0x7f97b019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b0103b00 0x7f97b019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 -- 172.21.15.19:0/685038147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0104d40 msgr2=0x7f97b019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0104d40 0x7f97b019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b67fe640 1 -- 172.21.15.19:0/685038147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b01a1d50 con 0x7f97b0109a40 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.632+0000 7f97b5ffd640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b0103b00 0x7f97b019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.633+0000 7f97b67fe640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b01a16e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005e30 tx=0x7f97a8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.633+0000 7f97a6ffd640 1 -- 172.21.15.19:0/685038147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8003b70 con 0x7f97b0109a40 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.633+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b01a1fe0 con 0x7f97b0109a40 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.633+0000 7f97a6ffd640 1 -- 172.21.15.19:0/685038147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a8003d10 con 0x7f97b0109a40 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.633+0000 7f97a6ffd640 1 -- 172.21.15.19:0/685038147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a802fac0 con 0x7f97b0109a40 2024-04-25T10:54:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.633+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b01a2440 con 0x7f97b0109a40 2024-04-25T10:54:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.634+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b006a160 con 0x7f97b0109a40 2024-04-25T10:54:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.635+0000 7f97a6ffd640 1 -- 172.21.15.19:0/685038147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f97a802fc60 con 0x7f97b0109a40 2024-04-25T10:54:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.636+0000 7f97a6ffd640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f979c0afc60 0x7f979c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.636+0000 7f97a6ffd640 1 -- 172.21.15.19:0/685038147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f97a8100580 con 0x7f97b0109a40 2024-04-25T10:54:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.636+0000 7f97b5ffd640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f979c0afc60 0x7f979c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:32.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.637+0000 7f97b5ffd640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f979c0afc60 0x7f979c0b2170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f97b0104700 tx=0x7f97ac005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:32.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.641+0000 7f97a6ffd640 1 -- 172.21.15.19:0/685038147 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97a80c8f60 con 0x7f97b0109a40 2024-04-25T10:54:32.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.866+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97b002db50 con 0x7f97b0109a40 2024-04-25T10:54:32.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.866+0000 7f97a6ffd640 1 -- 172.21.15.19:0/685038147 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f97a80c8860 con 0x7f97b0109a40 2024-04-25T10:54:32.867 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:54:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f979c0afc60 msgr2=0x7f979c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f979c0afc60 0x7f979c0b2170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f97b0104700 tx=0x7f97ac005f90 comp rx=0 tx=0).stop 2024-04-25T10:54:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 msgr2=0x7f97b01a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b01a16e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005e30 tx=0x7f97a8031840 comp rx=0 tx=0).stop 2024-04-25T10:54:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 shutdown_connections 2024-04-25T10:54:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f979c0afc60 0x7f979c0b2170 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97b0109a40 0x7f97b01a16e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0104d40 0x7f97b019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 --2- 172.21.15.19:0/685038147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b0103b00 0x7f97b019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 >> 172.21.15.19:0/685038147 conn(0x7f97b00fed10 msgr2=0x7f97b0105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 shutdown_connections 2024-04-25T10:54:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.869+0000 7f97b8261640 1 -- 172.21.15.19:0/685038147 wait complete. 2024-04-25T10:54:32.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:54:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.998+0000 7f65e4434640 1 -- 172.21.15.19:0/669022927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc108ea0 msgr2=0x7f65dc10b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.998+0000 7f65e4434640 1 --2- 172.21.15.19:0/669022927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc108ea0 0x7f65dc10b340 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009900 tx=0x7f65cc02a410 comp rx=0 tx=0).stop 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.998+0000 7f65e4434640 1 -- 172.21.15.19:0/669022927 shutdown_connections 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.998+0000 7f65e4434640 1 --2- 172.21.15.19:0/669022927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc10b9d0 0x7f65dc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.998+0000 7f65e4434640 1 --2- 172.21.15.19:0/669022927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc108ea0 0x7f65dc10b340 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.998+0000 7f65e4434640 1 --2- 172.21.15.19:0/669022927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65dc103fc0 0x7f65dc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.998+0000 7f65e4434640 1 -- 172.21.15.19:0/669022927 >> 172.21.15.19:0/669022927 conn(0x7f65dc0fd340 msgr2=0x7f65dc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 -- 172.21.15.19:0/669022927 shutdown_connections 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 -- 172.21.15.19:0/669022927 wait complete. 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 Processor -- start 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 -- start start 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc103fc0 0x7f65dc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65dc108ea0 0x7f65dc19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc10b9d0 0x7f65dc1a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc111870 con 0x7f65dc103fc0 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc1116f0 con 0x7f65dc10b9d0 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:32.999+0000 7f65e4434640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc1119f0 con 0x7f65dc108ea0 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc103fc0 0x7f65dc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc103fc0 0x7f65dc19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48948/0 (socket says 172.21.15.19:48948) 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 -- 172.21.15.19:0/4167033340 learned_addr learned my addr 172.21.15.19:0/4167033340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e19cf640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65dc108ea0 0x7f65dc19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e29d1640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc10b9d0 0x7f65dc1a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 -- 172.21.15.19:0/4167033340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65dc108ea0 msgr2=0x7f65dc19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65dc108ea0 0x7f65dc19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 -- 172.21.15.19:0/4167033340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc10b9d0 msgr2=0x7f65dc1a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc10b9d0 0x7f65dc1a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 -- 172.21.15.19:0/4167033340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc1a3f90 con 0x7f65dc103fc0 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e19cf640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65dc108ea0 0x7f65dc19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e29d1640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc10b9d0 0x7f65dc1a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65e21d0640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc103fc0 0x7f65dc19c450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e9f0 tx=0x7f65d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.000+0000 7f65d37fe640 1 -- 172.21.15.19:0/4167033340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d800ccc0 con 0x7f65dc103fc0 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.001+0000 7f65d37fe640 1 -- 172.21.15.19:0/4167033340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d80108e0 con 0x7f65dc103fc0 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.001+0000 7f65d37fe640 1 -- 172.21.15.19:0/4167033340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8018860 con 0x7f65dc103fc0 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.001+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc1a4280 con 0x7f65dc103fc0 2024-04-25T10:54:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.001+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a4750 con 0x7f65dc103fc0 2024-04-25T10:54:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.002+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc0fee60 con 0x7f65dc103fc0 2024-04-25T10:54:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.003+0000 7f65d37fe640 1 -- 172.21.15.19:0/4167033340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f65d8010430 con 0x7f65dc103fc0 2024-04-25T10:54:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.004+0000 7f65d37fe640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f65c80afc60 0x7f65c80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.005+0000 7f65e19cf640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f65c80afc60 0x7f65c80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.005+0000 7f65d37fe640 1 -- 172.21.15.19:0/4167033340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f65d80df550 con 0x7f65dc103fc0 2024-04-25T10:54:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.005+0000 7f65e19cf640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f65c80afc60 0x7f65c80b2170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004770 tx=0x7f65cc0314a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:33.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.009+0000 7f65d37fe640 1 -- 172.21.15.19:0/4167033340 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65d80a7e20 con 0x7f65dc103fc0 2024-04-25T10:54:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.202+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65dc0637a0 con 0x7f65dc103fc0 2024-04-25T10:54:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.203+0000 7f65d37fe640 1 -- 172.21.15.19:0/4167033340 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f65d80a7720 con 0x7f65dc103fc0 2024-04-25T10:54:33.204 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:54:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f65c80afc60 msgr2=0x7f65c80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f65c80afc60 0x7f65c80b2170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004770 tx=0x7f65cc0314a0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc103fc0 msgr2=0x7f65dc19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc103fc0 0x7f65dc19c450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e9f0 tx=0x7f65d800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 shutdown_connections 2024-04-25T10:54:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f65c80afc60 0x7f65c80b2170 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc10b9d0 0x7f65dc1a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65dc108ea0 0x7f65dc19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 --2- 172.21.15.19:0/4167033340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65dc103fc0 0x7f65dc19c450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 >> 172.21.15.19:0/4167033340 conn(0x7f65dc0fd340 msgr2=0x7f65dc109bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 shutdown_connections 2024-04-25T10:54:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.205+0000 7f65e4434640 1 -- 172.21.15.19:0/4167033340 wait complete. 2024-04-25T10:54:33.218 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/225163531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/153810321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:33.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:32 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/685038147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:33.220 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/225163531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/153810321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:33.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:32 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/685038147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/225163531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:33.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:33.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/153810321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:54:33.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:32 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/685038147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 -- 172.21.15.19:0/1700099207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c103c20 msgr2=0x7fac9c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 --2- 172.21.15.19:0/1700099207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c103c20 0x7fac9c104030 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fac88009900 tx=0x7fac8802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 -- 172.21.15.19:0/1700099207 shutdown_connections 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 --2- 172.21.15.19:0/1700099207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9c108920 0x7fac9c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 --2- 172.21.15.19:0/1700099207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c103c20 0x7fac9c104030 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 --2- 172.21.15.19:0/1700099207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c1029e0 0x7fac9c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 -- 172.21.15.19:0/1700099207 >> 172.21.15.19:0/1700099207 conn(0x7fac9c0fdbd0 msgr2=0x7fac9c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 -- 172.21.15.19:0/1700099207 shutdown_connections 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 -- 172.21.15.19:0/1700099207 wait complete. 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 Processor -- start 2024-04-25T10:54:33.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.335+0000 7faca3005640 1 -- start start 2024-04-25T10:54:33.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca3005640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c1029e0 0x7fac9c06bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca3005640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9c103c20 0x7fac9c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca3005640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c108920 0x7fac9c1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca3005640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10e4e0 con 0x7fac9c1029e0 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca3005640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10e360 con 0x7fac9c108920 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca3005640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10e660 con 0x7fac9c103c20 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c1029e0 0x7fac9c06bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c1029e0 0x7fac9c06bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48974/0 (socket says 172.21.15.19:48974) 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 -- 172.21.15.19:0/3087221439 learned_addr learned my addr 172.21.15.19:0/3087221439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca15a2640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c108920 0x7fac9c1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7fac9bfff640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9c103c20 0x7fac9c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 -- 172.21.15.19:0/3087221439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9c103c20 msgr2=0x7fac9c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9c103c20 0x7fac9c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 -- 172.21.15.19:0/3087221439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c108920 msgr2=0x7fac9c1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c108920 0x7fac9c1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 -- 172.21.15.19:0/3087221439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c1a1c50 con 0x7fac9c1029e0 2024-04-25T10:54:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7faca0da1640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c1029e0 0x7fac9c06bd20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c970 tx=0x7fac9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.336+0000 7fac99ffb640 1 -- 172.21.15.19:0/3087221439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90007c70 con 0x7fac9c1029e0 2024-04-25T10:54:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.337+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c1a1f40 con 0x7fac9c1029e0 2024-04-25T10:54:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.337+0000 7fac99ffb640 1 -- 172.21.15.19:0/3087221439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac90007e10 con 0x7fac9c1029e0 2024-04-25T10:54:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.337+0000 7fac99ffb640 1 -- 172.21.15.19:0/3087221439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90018760 con 0x7fac9c1029e0 2024-04-25T10:54:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.337+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c1a2400 con 0x7fac9c1029e0 2024-04-25T10:54:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.338+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c107730 con 0x7fac9c1029e0 2024-04-25T10:54:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.339+0000 7fac99ffb640 1 -- 172.21.15.19:0/3087221439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7fac9000fab0 con 0x7fac9c1029e0 2024-04-25T10:54:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.340+0000 7fac99ffb640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fac8c0afc60 0x7fac8c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.340+0000 7fac99ffb640 1 -- 172.21.15.19:0/3087221439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fac900df1e0 con 0x7fac9c1029e0 2024-04-25T10:54:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.340+0000 7fac9bfff640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fac8c0afc60 0x7fac8c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.341+0000 7fac9bfff640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fac8c0afc60 0x7fac8c0b2170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac88031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.344+0000 7fac99ffb640 1 -- 172.21.15.19:0/3087221439 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac90014030 con 0x7fac9c1029e0 2024-04-25T10:54:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.573+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac9c02db50 con 0x7fac9c1029e0 2024-04-25T10:54:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.574+0000 7fac99ffb640 1 -- 172.21.15.19:0/3087221439 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fac900a7ae0 con 0x7fac9c1029e0 2024-04-25T10:54:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:33.575 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:54:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:54:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fac8c0afc60 msgr2=0x7fac8c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fac8c0afc60 0x7fac8c0b2170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac88031400 comp rx=0 tx=0).stop 2024-04-25T10:54:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c1029e0 msgr2=0x7fac9c06bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c1029e0 0x7fac9c06bd20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c970 tx=0x7fac9000ce30 comp rx=0 tx=0).stop 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 shutdown_connections 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fac8c0afc60 0x7fac8c0b2170 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c108920 0x7fac9c1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9c103c20 0x7fac9c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.576+0000 7faca3005640 1 --2- 172.21.15.19:0/3087221439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac9c1029e0 0x7fac9c06bd20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.577+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 >> 172.21.15.19:0/3087221439 conn(0x7fac9c0fdbd0 msgr2=0x7fac9c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.577+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 shutdown_connections 2024-04-25T10:54:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.577+0000 7faca3005640 1 -- 172.21.15.19:0/3087221439 wait complete. 2024-04-25T10:54:33.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/865874147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b81029c0 msgr2=0x7fc7b81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/865874147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b81029c0 0x7fc7b81036c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009900 tx=0x7fc7a802a410 comp rx=0 tx=0).stop 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/865874147 shutdown_connections 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/865874147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7b8108900 0x7fc7b810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/865874147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7b8103c00 0x7fc7b8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/865874147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b81029c0 0x7fc7b81036c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/865874147 >> 172.21.15.19:0/865874147 conn(0x7fc7b80fdbd0 msgr2=0x7fc7b8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/865874147 shutdown_connections 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.706+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/865874147 wait complete. 2024-04-25T10:54:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 Processor -- start 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 -- start start 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7b81029c0 0x7fc7b806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7b8103c00 0x7fc7b819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8108900 0x7fc7b81a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810e510 con 0x7fc7b8108900 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810e390 con 0x7fc7b8103c00 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7bd7f3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810e690 con 0x7fc7b81029c0 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8108900 0x7fc7b81a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7b6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7b81029c0 0x7fc7b806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8108900 0x7fc7b81a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48982/0 (socket says 172.21.15.19:48982) 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7b77fe640 1 -- 172.21.15.19:0/2248855806 learned_addr learned my addr 172.21.15.19:0/2248855806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.707+0000 7fc7affff640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7b8103c00 0x7fc7b819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b77fe640 1 -- 172.21.15.19:0/2248855806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7b81029c0 msgr2=0x7fc7b806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b77fe640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7b81029c0 0x7fc7b806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b77fe640 1 -- 172.21.15.19:0/2248855806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7b8103c00 msgr2=0x7fc7b819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b77fe640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7b8103c00 0x7fc7b819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b77fe640 1 -- 172.21.15.19:0/2248855806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b81a1d30 con 0x7fc7b8108900 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b77fe640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8108900 0x7fc7b81a1530 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000eb10 tx=0x7fc7b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b4ff9640 1 -- 172.21.15.19:0/2248855806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0004b20 con 0x7fc7b8108900 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b4ff9640 1 -- 172.21.15.19:0/2248855806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0004cc0 con 0x7fc7b8108900 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b81a1fc0 con 0x7fc7b8108900 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7b4ff9640 1 -- 172.21.15.19:0/2248855806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0010670 con 0x7fc7b8108900 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.708+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a2480 con 0x7fc7b8108900 2024-04-25T10:54:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.709+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b8107710 con 0x7fc7b8108900 2024-04-25T10:54:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.711+0000 7fc7b4ff9640 1 -- 172.21.15.19:0/2248855806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7fc7b0005000 con 0x7fc7b8108900 2024-04-25T10:54:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.712+0000 7fc7b4ff9640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc7a40afc60 0x7fc7a40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.712+0000 7fc7b6ffd640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc7a40afc60 0x7fc7a40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.712+0000 7fc7b4ff9640 1 -- 172.21.15.19:0/2248855806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc7b00df750 con 0x7fc7b8108900 2024-04-25T10:54:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.712+0000 7fc7b6ffd640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc7a40afc60 0x7fc7a40b2170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc7a802aa00 tx=0x7fc7a80314a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:33.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.718+0000 7fc7b4ff9640 1 -- 172.21.15.19:0/2248855806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7b0053ff0 con 0x7fc7b8108900 2024-04-25T10:54:33.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.924+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7b80637a0 con 0x7fc7b8108900 2024-04-25T10:54:33.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.931+0000 7fc7b4ff9640 1 -- 172.21.15.19:0/2248855806 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23035 (secure 0 0 0) 0x7fc7b00a8020 con 0x7fc7b8108900 2024-04-25T10:54:33.932 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:54:33.933 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T10:54:29.901359+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/48031724":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/2535542967":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/4264937365":"2024-04-26T10:54:29.901303+0000","172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3805408646":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6800/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:54:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.933+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc7a40afc60 msgr2=0x7fc7a40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.933+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc7a40afc60 0x7fc7a40b2170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc7a802aa00 tx=0x7fc7a80314a0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8108900 msgr2=0x7fc7b81a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8108900 0x7fc7b81a1530 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000eb10 tx=0x7fc7b000c2d0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 shutdown_connections 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc7a40afc60 0x7fc7a40b2170 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8108900 0x7fc7b81a1530 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7b8103c00 0x7fc7b819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 --2- 172.21.15.19:0/2248855806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7b81029c0 0x7fc7b806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 >> 172.21.15.19:0/2248855806 conn(0x7fc7b80fdbd0 msgr2=0x7fc7b81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 shutdown_connections 2024-04-25T10:54:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:33.934+0000 7fc7bd7f3640 1 -- 172.21.15.19:0/2248855806 wait complete. 2024-04-25T10:54:33.947 DEBUG:tasks.fwd_scrub:Yielding 2024-04-25T10:54:33.947 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T10:54:33.957 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T10:54:33.957 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:54:33.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-04-25T10:54:33.958 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-25T10:54:33.958 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:54:33.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:54:33.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:54:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.086+0000 7f63ec73f640 1 -- 172.21.15.19:0/2715945717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4105090 msgr2=0x7f63e41054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.086+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2715945717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4105090 0x7f63e41054a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009900 tx=0x7f63e002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.086+0000 7f63ec73f640 1 -- 172.21.15.19:0/2715945717 shutdown_connections 2024-04-25T10:54:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.086+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2715945717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e4102240 0x7f63e410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.086+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2715945717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e4101740 0x7f63e4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.086+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2715945717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4105090 0x7f63e41054a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.086+0000 7f63ec73f640 1 -- 172.21.15.19:0/2715945717 >> 172.21.15.19:0/2715945717 conn(0x7f63e40fd340 msgr2=0x7f63e40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.087+0000 7f63ec73f640 1 -- 172.21.15.19:0/2715945717 shutdown_connections 2024-04-25T10:54:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.087+0000 7f63ec73f640 1 -- 172.21.15.19:0/2715945717 wait complete. 2024-04-25T10:54:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.087+0000 7f63ec73f640 1 Processor -- start 2024-04-25T10:54:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ec73f640 1 -- start start 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ec73f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e4101740 0x7f63e419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ec73f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4102240 0x7f63e419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ec73f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e4105090 0x7f63e41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ec73f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63e410f5e0 con 0x7f63e4102240 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ec73f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63e410f460 con 0x7f63e4105090 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ec73f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63e410f760 con 0x7f63e4101740 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63e9cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4102240 0x7f63e419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63eacdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e4105090 0x7f63e41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63e9cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4102240 0x7f63e419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49002/0 (socket says 172.21.15.19:49002) 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63e9cda640 1 -- 172.21.15.19:0/2628360439 learned_addr learned my addr 172.21.15.19:0/2628360439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63ea4db640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e4101740 0x7f63e419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63eacdc640 1 -- 172.21.15.19:0/2628360439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e4101740 msgr2=0x7f63e419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63eacdc640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e4101740 0x7f63e419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63eacdc640 1 -- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4102240 msgr2=0x7f63e419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63eacdc640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4102240 0x7f63e419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.088+0000 7f63eacdc640 1 -- 172.21.15.19:0/2628360439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41a1d60 con 0x7f63e4105090 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.089+0000 7f63eacdc640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e4105090 0x7f63e41a16f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009fd0 tx=0x7f63dc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.089+0000 7f63db7fe640 1 -- 172.21.15.19:0/2628360439 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00ce20 con 0x7f63e4105090 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.089+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41a1ff0 con 0x7f63e4105090 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.089+0000 7f63db7fe640 1 -- 172.21.15.19:0/2628360439 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63dc005020 con 0x7f63e4105090 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.089+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41a24b0 con 0x7f63e4105090 2024-04-25T10:54:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.089+0000 7f63db7fe640 1 -- 172.21.15.19:0/2628360439 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc010690 con 0x7f63e4105090 2024-04-25T10:54:34.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.091+0000 7f63db7fe640 1 -- 172.21.15.19:0/2628360439 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f63dc0054d0 con 0x7f63e4105090 2024-04-25T10:54:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.092+0000 7f63f35f2740 1 Processor -- start 2024-04-25T10:54:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63f35f2740 1 -- start start 2024-04-25T10:54:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63db7fe640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f63d00afc60 0x7f63d00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561999886550 0x561999c50a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199988e730 0x561999c57920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63f35f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561999bb5b00 con 0x5619998af410 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63f35f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561999b51920 con 0x56199988e730 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63f35f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561999bb1ac0 con 0x561999886550 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63db7fe640 1 -- 172.21.15.19:0/2628360439 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f63dc0df6c0 con 0x7f63e4105090 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49004/0 (socket says 172.21.15.19:49004) 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 -- 172.21.15.19:0/2843602340 learned_addr learned my addr 172.21.15.19:0/2843602340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63e9cda640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561999886550 0x561999c50a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63eacdc640 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199988e730 0x561999c57920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63e9cda640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f63d00afc60 0x7f63d00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 -- 172.21.15.19:0/2843602340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561999886550 msgr2=0x561999c50a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561999886550 0x561999c50a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 -- 172.21.15.19:0/2843602340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199988e730 msgr2=0x561999c57920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199988e730 0x561999c57920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63ea4db640 1 -- 172.21.15.19:0/2843602340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199998f7e0 con 0x5619998af410 2024-04-25T10:54:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.093+0000 7f63eacdc640 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199988e730 0x561999c57920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.094+0000 7f63ea4db640 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c504a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009230 tx=0x7f63e0031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b4996ec8baea8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.094+0000 7f63e9cda640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f63d00afc60 0x7f63d00b2170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x561999c51880 tx=0x7f63d400b4a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.094+0000 7f63d8ff9640 1 -- 172.21.15.19:0/2843602340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0035030 con 0x5619998af410 2024-04-25T10:54:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.094+0000 7f63d8ff9640 1 -- 172.21.15.19:0/2843602340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0040070 con 0x5619998af410 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.094+0000 7f63d8ff9640 1 -- 172.21.15.19:0/2843602340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0003850 con 0x5619998af410 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 -- 172.21.15.19:0/2843602340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 msgr2=0x561999c504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c504a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009230 tx=0x7f63e0031b10 comp rx=0 tx=0).stop 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 -- 172.21.15.19:0/2843602340 shutdown_connections 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199988e730 0x561999c57920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561999886550 0x561999c50a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 --2- 172.21.15.19:0/2843602340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c504a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 -- 172.21.15.19:0/2843602340 >> 172.21.15.19:0/2843602340 conn(0x5619998a3560 msgr2=0x561999c4be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 -- 172.21.15.19:0/2843602340 shutdown_connections 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 -- 172.21.15.19:0/2843602340 wait complete. 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.095+0000 7f63f35f2740 1 -- start start 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 1 Processor -- start 2024-04-25T10:54:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c65b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561999c5c720 con 0x5619998af410 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561999b4f5d0 con 0x561999886550 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561999c5c8a0 con 0x56199988e730 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63f35f2740 10 client.0 _ll_register_callbacks cb 0x561999b3d080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c65b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63e9cda640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c65b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49020/0 (socket says 172.21.15.19:49020) 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 -- 172.21.15.19:0/4291943388 learned_addr learned my addr 172.21.15.19:0/4291943388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63ea4db640 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 -- 172.21.15.19:0/4291943388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 msgr2=0x561999c5ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c5ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 -- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 msgr2=0x561999c5e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c5e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.096+0000 7f63eacdc640 1 -- 172.21.15.19:0/4291943388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619999eff70 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63e9cda640 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c5ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63eacdc640 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c65b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0a77c0 tx=0x7f63dc0a7470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5096bd566f3d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63cdffb640 1 -- 172.21.15.19:0/4291943388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0abbe0 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63cdffb640 1 -- 172.21.15.19:0/4291943388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f63dc090110 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e0040070 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63cdffb640 1 -- 172.21.15.19:0/4291943388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63dc090310 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63f35f2740 10 client.15375 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561999887170 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.097+0000 7f63cdffb640 1 -- 172.21.15.19:0/4291943388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0040070 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63cdffb640 1 -- 172.21.15.19:0/4291943388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f63dc0c8ae0 con 0x5619998af410 2024-04-25T10:54:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63f35f2740 10 client.15375 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63f35f2740 10 client.15375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63f35f2740 20 client.15375 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63f35f2740 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56199986e1f0 0x561999c6c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63f35f2740 4 client.15375 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561999bc3710 con 0x56199986e1f0 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.098+0000 7f63ea4db640 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56199986e1f0 0x561999c6c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.099+0000 7f63ea4db640 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56199986e1f0 0x561999c6c010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35ed59860b98cc3 server_cookie=85836c9c58938d96 in_seq=0 out_seq=0 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.099+0000 7f63cdffb640 10 client.15375 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.099+0000 7f63cdffb640 1 -- 172.21.15.19:0/4291943388 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561999bc3710 con 0x56199986e1f0 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.099+0000 7f63cdffb640 10 client.15375 handle_command_reply: tid=0 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.099+0000 7f63f35f2740 1 client.15375 shutdown 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.099+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 msgr2=0x561999c65b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.099+0000 7f63f35f2740 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c65b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0a77c0 tx=0x7f63dc0a7470 comp rx=0 tx=0).stop 2024-04-25T10:54:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 shutdown_connections 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56199986e1f0 0x561999c6c010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619998af410 0x561999c65b10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c5ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63cdffb640 0 client.15375 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 --2- 172.21.15.19:0/4291943388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 >> 172.21.15.19:0/4291943388 conn(0x5619998a3560 msgr2=0x561999c4aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 shutdown_connections 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 1 -- 172.21.15.19:0/4291943388 wait complete. 2024-04-25T10:54:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.100+0000 7f63f35f2740 20 client.15375 trim_cache size 0 max 16384 2024-04-25T10:54:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.104+0000 7f27782af640 1 -- 172.21.15.19:0/603324579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770104010 msgr2=0x7f2770104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.104+0000 7f27782af640 1 --2- 172.21.15.19:0/603324579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770104010 0x7f2770104420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f276c009900 tx=0x7f276c02a410 comp rx=0 tx=0).stop 2024-04-25T10:54:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 -- 172.21.15.19:0/603324579 shutdown_connections 2024-04-25T10:54:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 --2- 172.21.15.19:0/603324579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f277010ba20 0x7f277010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 --2- 172.21.15.19:0/603324579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2770108ef0 0x7f277010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 --2- 172.21.15.19:0/603324579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770104010 0x7f2770104420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 -- 172.21.15.19:0/603324579 >> 172.21.15.19:0/603324579 conn(0x7f27700fd390 msgr2=0x7f27700ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 -- 172.21.15.19:0/603324579 shutdown_connections 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 -- 172.21.15.19:0/603324579 wait complete. 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 Processor -- start 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.105+0000 7f27782af640 1 -- start start 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f27782af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770104010 0x7f277019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f27782af640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2770108ef0 0x7f277019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f27782af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010ba20 0x7f27701a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f27782af640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2770111830 con 0x7f2770104010 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f27782af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27701116b0 con 0x7f277010ba20 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f27782af640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27701119b0 con 0x7f2770108ef0 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770104010 0x7f277019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770104010 0x7f277019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49040/0 (socket says 172.21.15.19:49040) 2024-04-25T10:54:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 -- 172.21.15.19:0/2893572214 learned_addr learned my addr 172.21.15.19:0/2893572214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277584a640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2770108ef0 0x7f277019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277684c640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010ba20 0x7f27701a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 -- 172.21.15.19:0/2893572214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2770108ef0 msgr2=0x7f277019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2770108ef0 0x7f277019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 -- 172.21.15.19:0/2893572214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010ba20 msgr2=0x7f27701a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010ba20 0x7f27701a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 -- 172.21.15.19:0/2893572214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27701a3f80 con 0x7f2770104010 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.106+0000 7f277604b640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770104010 0x7f277019c440 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f276c006010 tx=0x7f276c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.107+0000 7f27677fe640 1 -- 172.21.15.19:0/2893572214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c03f030 con 0x7f2770104010 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.107+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27701a4210 con 0x7f2770104010 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.107+0000 7f27677fe640 1 -- 172.21.15.19:0/2893572214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f276c003aa0 con 0x7f2770104010 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.107+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27701a4750 con 0x7f2770104010 2024-04-25T10:54:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.107+0000 7f27677fe640 1 -- 172.21.15.19:0/2893572214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c035070 con 0x7f2770104010 2024-04-25T10:54:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.108+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27700feeb0 con 0x7f2770104010 2024-04-25T10:54:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.110+0000 7f27677fe640 1 -- 172.21.15.19:0/2893572214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f276c042030 con 0x7f2770104010 2024-04-25T10:54:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.111+0000 7f27677fe640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f275c0afc50 0x7f275c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.111+0000 7f277584a640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f275c0afc50 0x7f275c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.111+0000 7f27677fe640 1 -- 172.21.15.19:0/2893572214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f276c0ffa30 con 0x7f2770104010 2024-04-25T10:54:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.112+0000 7f277584a640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f275c0afc50 0x7f275c0b2160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f277019d4e0 tx=0x7f276000b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.115+0000 7f27677fe640 1 -- 172.21.15.19:0/2893572214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f276c0c7cb0 con 0x7f2770104010 2024-04-25T10:54:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 Processor -- start 2024-04-25T10:54:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 -- start start 2024-04-25T10:54:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199986e1f0 0x561999c54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561999886550 0x561999c65a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c4f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41a7210 con 0x561999886550 2024-04-25T10:54:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41a9a90 con 0x56199986e1f0 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.121+0000 7f63f35f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41a7880 con 0x56199988e730 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63e9cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561999886550 0x561999c65a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63ea4db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199986e1f0 0x561999c54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63e9cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561999886550 0x561999c65a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49048/0 (socket says 172.21.15.19:49048) 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63e9cda640 1 -- 172.21.15.19:0/426281245 learned_addr learned my addr 172.21.15.19:0/426281245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63eacdc640 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c4f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63ea4db640 1 -- 172.21.15.19:0/426281245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 msgr2=0x561999c4f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63ea4db640 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c4f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63ea4db640 1 -- 172.21.15.19:0/426281245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561999886550 msgr2=0x561999c65a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63ea4db640 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561999886550 0x561999c65a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63ea4db640 1 -- 172.21.15.19:0/426281245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561999c57ac0 con 0x56199986e1f0 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63eacdc640 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c4f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63e9cda640 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561999886550 0x561999c65a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63ea4db640 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199986e1f0 0x561999c54d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63e002a930 tx=0x7f63e0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5d9ef49968caa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63cffff640 1 -- 172.21.15.19:0/426281245 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0035040 con 0x56199986e1f0 2024-04-25T10:54:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63cffff640 1 -- 172.21.15.19:0/426281245 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0040050 con 0x56199986e1f0 2024-04-25T10:54:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.122+0000 7f63cffff640 1 -- 172.21.15.19:0/426281245 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0051440 con 0x56199986e1f0 2024-04-25T10:54:34.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 -- 172.21.15.19:0/426281245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199986e1f0 msgr2=0x561999c54d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199986e1f0 0x561999c54d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63e002a930 tx=0x7f63e0005fd0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 -- 172.21.15.19:0/426281245 shutdown_connections 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999c4f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561999886550 0x561999c65a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 --2- 172.21.15.19:0/426281245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56199986e1f0 0x561999c54d70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 -- 172.21.15.19:0/426281245 >> 172.21.15.19:0/426281245 conn(0x5619998a3560 msgr2=0x561999c4ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 -- 172.21.15.19:0/426281245 shutdown_connections 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.123+0000 7f63f35f2740 1 -- 172.21.15.19:0/426281245 wait complete. 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 -- start start 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 Processor -- start 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56199986e1f0 0x561999c650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c65610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999cd2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41a7f50 con 0x56199986e1f0 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41a93c0 con 0x561999886550 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41a8620 con 0x56199988e730 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63f35f2740 10 client.0 _ll_register_callbacks cb 0x7f63dc005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63ea4db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56199986e1f0 0x561999c650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63ea4db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56199986e1f0 0x561999c650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49060/0 (socket says 172.21.15.19:49060) 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63ea4db640 1 -- 172.21.15.19:0/3690063202 learned_addr learned my addr 172.21.15.19:0/3690063202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63e9cda640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c65610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.124+0000 7f63eacdc640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999cd2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63ea4db640 1 -- 172.21.15.19:0/3690063202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 msgr2=0x561999cd2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63ea4db640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999cd2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63ea4db640 1 -- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 msgr2=0x561999c65610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63ea4db640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c65610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63ea4db640 1 -- 172.21.15.19:0/3690063202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561999cd2e50 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63eacdc640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999cd2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63ea4db640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56199986e1f0 0x561999c650d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f63e0005ab0 tx=0x7f63e00314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4207950e77788a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63cdffb640 1 -- 172.21.15.19:0/3690063202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0035040 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63cdffb640 1 -- 172.21.15.19:0/3690063202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f63e0045cc0 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e0040050 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63cdffb640 1 -- 172.21.15.19:0/3690063202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63e0031820 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63f35f2740 10 client.15387 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561999cd4110 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.125+0000 7f63cdffb640 1 -- 172.21.15.19:0/3690063202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0040050 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63cdffb640 1 -- 172.21.15.19:0/3690063202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f63e004e070 con 0x56199986e1f0 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63f35f2740 10 client.15387 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63f35f2740 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63f35f2740 20 client.15387 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63f35f2740 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5619998af410 0x561999cd66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63f35f2740 4 client.15387 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561999bbd580 con 0x5619998af410 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63e9cda640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5619998af410 0x561999cd66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63cdffb640 10 client.15387 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.126+0000 7f63e9cda640 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5619998af410 0x561999cd66b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9019cbc2dabc0eb9 server_cookie=5448e4d9aa2ed06c in_seq=0 out_seq=0 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.127+0000 7f63cdffb640 1 -- 172.21.15.19:0/3690063202 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561999bbd580 con 0x5619998af410 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.127+0000 7f63cdffb640 10 client.15387 handle_command_reply: tid=0 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.127+0000 7f63f35f2740 1 client.15387 shutdown 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56199986e1f0 msgr2=0x561999c650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56199986e1f0 0x561999c650d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f63e0005ab0 tx=0x7f63e00314c0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 shutdown_connections 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5619998af410 0x561999cd66b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63cdffb640 0 client.15387 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56199988e730 0x561999cd2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561999886550 0x561999c65610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 --2- 172.21.15.19:0/3690063202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56199986e1f0 0x561999c650d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 >> 172.21.15.19:0/3690063202 conn(0x5619998a3560 msgr2=0x561999c4f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 shutdown_connections 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 1 -- 172.21.15.19:0/3690063202 wait complete. 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.128+0000 7f63f35f2740 20 client.15387 trim_cache size 0 max 16384 2024-04-25T10:54:34.134 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:54:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:54:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "a8699705-fabe-4f32-a5b0-3fe4f86e0e24", 2024-04-25T10:54:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:54:34.135 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f63d00afc60 msgr2=0x7f63d00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f63d00afc60 0x7f63d00b2170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x561999c51880 tx=0x7f63d400b4a0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e4105090 msgr2=0x7f63e41a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e4105090 0x7f63e41a16f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009fd0 tx=0x7f63dc00ef10 comp rx=0 tx=0).stop 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 shutdown_connections 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f63d00afc60 0x7f63d00b2170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e4105090 0x7f63e41a16f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4102240 0x7f63e419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 --2- 172.21.15.19:0/2628360439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e4101740 0x7f63e419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 >> 172.21.15.19:0/2628360439 conn(0x7f63e40fd340 msgr2=0x7f63e4105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 shutdown_connections 2024-04-25T10:54:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.129+0000 7f63ec73f640 1 -- 172.21.15.19:0/2628360439 wait complete. 2024-04-25T10:54:34.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:54:34.141 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:54:34.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4167033340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:34.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: Unable to set osd_memory_target on smithi112 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3087221439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:54:34.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:33 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:54:34.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 -- 172.21.15.19:0/3748620129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9500103c20 msgr2=0x7f9500104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 --2- 172.21.15.19:0/3748620129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9500103c20 0x7f9500104030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009900 tx=0x7f94f002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 -- 172.21.15.19:0/3748620129 shutdown_connections 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 --2- 172.21.15.19:0/3748620129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9500108920 0x7f950010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 --2- 172.21.15.19:0/3748620129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9500103c20 0x7f9500104030 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 --2- 172.21.15.19:0/3748620129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95001029e0 0x7f95001036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 -- 172.21.15.19:0/3748620129 >> 172.21.15.19:0/3748620129 conn(0x7f95000fdbd0 msgr2=0x7f9500100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 -- 172.21.15.19:0/3748620129 shutdown_connections 2024-04-25T10:54:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.258+0000 7f9507bf3640 1 -- 172.21.15.19:0/3748620129 wait complete. 2024-04-25T10:54:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 Processor -- start 2024-04-25T10:54:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 -- start start 2024-04-25T10:54:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95001029e0 0x7f9500199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9500103c20 0x7f9500199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9500108920 0x7f95001a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f950010e4e0 con 0x7f95001029e0 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f950010e360 con 0x7f9500103c20 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9507bf3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f950010e660 con 0x7f9500108920 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f950598f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95001029e0 0x7f9500199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f950598f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95001029e0 0x7f9500199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49078/0 (socket says 172.21.15.19:49078) 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f950598f640 1 -- 172.21.15.19:0/55302405 learned_addr learned my addr 172.21.15.19:0/55302405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f950518e640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9500103c20 0x7f9500199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.259+0000 7f9506190640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9500108920 0x7f95001a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f950518e640 1 -- 172.21.15.19:0/55302405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9500108920 msgr2=0x7f95001a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f950518e640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9500108920 0x7f95001a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f950518e640 1 -- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95001029e0 msgr2=0x7f9500199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f950518e640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95001029e0 0x7f9500199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f950518e640 1 -- 172.21.15.19:0/55302405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95001a0f50 con 0x7f9500103c20 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f950598f640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95001029e0 0x7f9500199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f9506190640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9500108920 0x7f95001a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f950518e640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9500103c20 0x7f9500199950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006010 tx=0x7f94f0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f94f6ffd640 1 -- 172.21.15.19:0/55302405 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0035050 con 0x7f9500103c20 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9500069e90 con 0x7f9500103c20 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f94f6ffd640 1 -- 172.21.15.19:0/55302405 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f0031910 con 0x7f9500103c20 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f94f6ffd640 1 -- 172.21.15.19:0/55302405 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f00389d0 con 0x7f9500103c20 2024-04-25T10:54:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.260+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950006a380 con 0x7f9500103c20 2024-04-25T10:54:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.263+0000 7f94f6ffd640 1 -- 172.21.15.19:0/55302405 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f94f0003ce0 con 0x7f9500103c20 2024-04-25T10:54:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.263+0000 7f950eb79740 1 Processor -- start 2024-04-25T10:54:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.263+0000 7f950eb79740 1 -- start start 2024-04-25T10:54:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f950eb79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26073e390 0x55d260adfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f950eb79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2607154d0 0x55d260adffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f950eb79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260ae0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f950eb79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d260a44d80 con 0x55d2607154d0 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f950eb79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d2609fe8b0 con 0x55d26073e390 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f950eb79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d260a40d40 con 0x55d26071d6b0 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f9506190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26073e390 0x55d260adfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f950598f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2607154d0 0x55d260adffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f9506190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26073e390 0x55d260adfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:59588/0 (socket says 172.21.15.19:59588) 2024-04-25T10:54:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.264+0000 7f9506190640 1 -- 172.21.15.19:0/1063107425 learned_addr learned my addr 172.21.15.19:0/1063107425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f94f6ffd640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f94ec0afc60 0x7f94ec0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f9506190640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f94ec0afc60 0x7f94ec0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f9506190640 1 -- 172.21.15.19:0/1063107425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 msgr2=0x55d260ae0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f950518e640 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260ae0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f9506190640 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260ae0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f9506190640 1 -- 172.21.15.19:0/1063107425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2607154d0 msgr2=0x55d260adffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f9506190640 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2607154d0 0x55d260adffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f9506190640 1 -- 172.21.15.19:0/1063107425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2609cfa40 con 0x55d26073e390 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f950598f640 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2607154d0 0x55d260adffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f94f6ffd640 1 -- 172.21.15.19:0/55302405 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f94f0100ad0 con 0x7f9500103c20 2024-04-25T10:54:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f9506190640 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26073e390 0x55d260adfa60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009fd0 tx=0x7f94f8013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72ed81b8c23e740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.265+0000 7f94ebfff640 1 -- 172.21.15.19:0/1063107425 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8022020 con 0x55d26073e390 2024-04-25T10:54:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.266+0000 7f94ebfff640 1 -- 172.21.15.19:0/1063107425 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f8004620 con 0x55d26073e390 2024-04-25T10:54:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.266+0000 7f94ebfff640 1 -- 172.21.15.19:0/1063107425 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f801ee50 con 0x55d26073e390 2024-04-25T10:54:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.266+0000 7f950eb79740 1 -- 172.21.15.19:0/1063107425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26073e390 msgr2=0x55d260adfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.266+0000 7f950eb79740 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26073e390 0x55d260adfa60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009fd0 tx=0x7f94f8013830 comp rx=0 tx=0).stop 2024-04-25T10:54:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.266+0000 7f9506190640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f94ec0afc60 0x7f94ec0b2170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94f8023000 tx=0x7f94f8021000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 -- 172.21.15.19:0/1063107425 shutdown_connections 2024-04-25T10:54:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260ae0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2607154d0 0x55d260adffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 --2- 172.21.15.19:0/1063107425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26073e390 0x55d260adfa60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 -- 172.21.15.19:0/1063107425 >> 172.21.15.19:0/1063107425 conn(0x55d2607324e0 msgr2=0x55d260adb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 -- 172.21.15.19:0/1063107425 shutdown_connections 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 -- 172.21.15.19:0/1063107425 wait complete. 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 -- start start 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.267+0000 7f950eb79740 1 Processor -- start 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950eb79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260aede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950eb79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260aee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950eb79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950eb79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d260ae1780 con 0x55d26073e390 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950eb79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d2609cbd10 con 0x55d2607154d0 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950eb79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d260ae1900 con 0x55d26071d6b0 2024-04-25T10:54:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950eb79740 10 client.0 _ll_register_callbacks cb 0x55d2609f8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950518e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260aee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49106/0 (socket says 172.21.15.19:49106) 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950518e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260aee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:53852/0 (socket says 172.21.15.19:53852) 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 -- 172.21.15.19:0/130947745 learned_addr learned my addr 172.21.15.19:0/130947745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950598f640 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260aede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 -- 172.21.15.19:0/130947745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 msgr2=0x55d260aee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260aee360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 -- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 msgr2=0x55d260aede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260aede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 -- 172.21.15.19:0/130947745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2608207d0 con 0x55d26073e390 2024-04-25T10:54:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f950518e640 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260aee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.268+0000 7f9506190640 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af51e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005c40 tx=0x7f94f8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04704ff4c1b414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f94e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f94e97fa640 1 -- 172.21.15.19:0/130947745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8035060 con 0x55d26073e390 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f8004620 con 0x55d26073e390 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f94e97fa640 1 -- 172.21.15.19:0/130947745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f94f801e610 con 0x55d26073e390 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f94e97fa640 1 -- 172.21.15.19:0/130947745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f94f8022040 con 0x55d26073e390 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f950eb79740 10 client.15399 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d2607563f0 con 0x55d26073e390 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f94e97fa640 1 -- 172.21.15.19:0/130947745 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f8004620 con 0x55d26073e390 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f94e97fa640 1 -- 172.21.15.19:0/130947745 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f94f8027420 con 0x55d26073e390 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f950eb79740 10 client.15399 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f950eb79740 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.269+0000 7f950eb79740 20 client.15399 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.270+0000 7f950eb79740 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606fcf60 0x55d260afb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.270+0000 7f950eb79740 4 client.15399 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.270+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d260a52990 con 0x55d2606fcf60 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.270+0000 7f950598f640 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606fcf60 0x55d260afb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.270+0000 7f94e97fa640 10 client.15399 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.270+0000 7f950598f640 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606fcf60 0x55d260afb4e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc037f072307634f server_cookie=d42595b60e7cd2b5 in_seq=0 out_seq=0 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f94e97fa640 1 -- 172.21.15.19:0/130947745 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d260a52990 con 0x55d2606fcf60 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f94e97fa640 10 client.15399 handle_command_reply: tid=0 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 client.15399 shutdown 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 msgr2=0x55d260af51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af51e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005c40 tx=0x7f94f8004530 comp rx=0 tx=0).stop 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 shutdown_connections 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606fcf60 0x55d260afb4e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af51e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260aee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f94e97fa640 0 client.15399 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 --2- 172.21.15.19:0/130947745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260aede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 >> 172.21.15.19:0/130947745 conn(0x55d2607324e0 msgr2=0x55d260ada3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 shutdown_connections 2024-04-25T10:54:34.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 1 -- 172.21.15.19:0/130947745 wait complete. 2024-04-25T10:54:34.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.271+0000 7f950eb79740 20 client.15399 trim_cache size 0 max 16384 2024-04-25T10:54:34.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 Processor -- start 2024-04-25T10:54:34.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 -- start start 2024-04-25T10:54:34.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260adaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260adb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95001a72c0 con 0x55d26073e390 2024-04-25T10:54:34.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95001a9b40 con 0x55d2607154d0 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f950eb79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95001a7930 con 0x55d26071d6b0 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f9506190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f9506190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49122/0 (socket says 172.21.15.19:49122) 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.290+0000 7f9506190640 1 -- 172.21.15.19:0/1015681475 learned_addr learned my addr 172.21.15.19:0/1015681475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f950598f640 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260adaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f950518e640 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260adb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f9506190640 1 -- 172.21.15.19:0/1015681475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 msgr2=0x55d260adb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f9506190640 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260adb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f9506190640 1 -- 172.21.15.19:0/1015681475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 msgr2=0x55d260adaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f9506190640 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260adaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f9506190640 1 -- 172.21.15.19:0/1015681475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2607160f0 con 0x55d26073e390 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f9506190640 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af4ef0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f94f8004500 tx=0x7f94f8003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d568db78d8a1c75f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f94eb7fe640 1 -- 172.21.15.19:0/1015681475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8035070 con 0x55d26073e390 2024-04-25T10:54:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f94eb7fe640 1 -- 172.21.15.19:0/1015681475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f8027420 con 0x55d26073e390 2024-04-25T10:54:34.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.291+0000 7f94eb7fe640 1 -- 172.21.15.19:0/1015681475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f801e8e0 con 0x55d26073e390 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 -- 172.21.15.19:0/1015681475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 msgr2=0x55d260af4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af4ef0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f94f8004500 tx=0x7f94f8003f80 comp rx=0 tx=0).stop 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 -- 172.21.15.19:0/1015681475 shutdown_connections 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260af4ef0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d26071d6b0 0x55d260adb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 --2- 172.21.15.19:0/1015681475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2607154d0 0x55d260adaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 -- 172.21.15.19:0/1015681475 >> 172.21.15.19:0/1015681475 conn(0x55d2607324e0 msgr2=0x55d260ae4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 -- 172.21.15.19:0/1015681475 shutdown_connections 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.292+0000 7f950eb79740 1 -- 172.21.15.19:0/1015681475 wait complete. 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 -- start start 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 Processor -- start 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2607154d0 0x55d260aeffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26071d6b0 0x55d260af04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260b61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95001a8000 con 0x55d26073e390 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95001a9470 con 0x55d26071d6b0 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95001a86d0 con 0x55d2607154d0 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950eb79740 10 client.0 _ll_register_callbacks cb 0x7f94fc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f9506190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260b61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f9506190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260b61880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49124/0 (socket says 172.21.15.19:49124) 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f9506190640 1 -- 172.21.15.19:0/1355641267 learned_addr learned my addr 172.21.15.19:0/1355641267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950598f640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2607154d0 0x55d260aeffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.293+0000 7f950518e640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26071d6b0 0x55d260af04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f9506190640 1 -- 172.21.15.19:0/1355641267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2607154d0 msgr2=0x55d260aeffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f9506190640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2607154d0 0x55d260aeffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f9506190640 1 -- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26071d6b0 msgr2=0x55d260af04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f9506190640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26071d6b0 0x55d260af04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f9506190640 1 -- 172.21.15.19:0/1355641267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d260a054b0 con 0x55d26073e390 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f950598f640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2607154d0 0x55d260aeffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f950518e640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26071d6b0 0x55d260af04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f9506190640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260b61880 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009690 tx=0x7f94f8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208324210165f930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f94e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f94e97fa640 1 -- 172.21.15.19:0/1355641267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8035070 con 0x55d26073e390 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f94e97fa640 1 -- 172.21.15.19:0/1355641267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f94f8044480 con 0x55d26073e390 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f8027420 con 0x55d26073e390 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f94e97fa640 1 -- 172.21.15.19:0/1355641267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f94f8052050 con 0x55d26073e390 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f950eb79740 10 client.15405 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d260b62bc0 con 0x55d26073e390 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.294+0000 7f94e97fa640 1 -- 172.21.15.19:0/1355641267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f8027420 con 0x55d26073e390 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f94e97fa640 1 -- 172.21.15.19:0/1355641267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f94f8042020 con 0x55d26073e390 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950eb79740 10 client.15405 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950eb79740 10 client.15405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950eb79740 20 client.15405 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950eb79740 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606d23a0 0x55d260b65520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950eb79740 4 client.15405 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d260a4c800 con 0x55d2606d23a0 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950598f640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606d23a0 0x55d260b65520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f94e97fa640 10 client.15405 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.295+0000 7f950598f640 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606d23a0 0x55d260b65520 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc5c86ffcc00e2c server_cookie=dd6f2bffe5aa2ba0 in_seq=0 out_seq=0 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f94e97fa640 1 -- 172.21.15.19:0/1355641267 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d260a4c800 con 0x55d2606d23a0 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f94e97fa640 10 client.15405 handle_command_reply: tid=0 2024-04-25T10:54:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 client.15405 shutdown 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 msgr2=0x55d260b61880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260b61880 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009690 tx=0x7f94f8005770 comp rx=0 tx=0).stop 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 shutdown_connections 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55d2606d23a0 0x55d260b65520 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d26073e390 0x55d260b61880 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f94e97fa640 0 client.15405 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d26071d6b0 0x55d260af04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 --2- 172.21.15.19:0/1355641267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2607154d0 0x55d260aeffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 >> 172.21.15.19:0/1355641267 conn(0x55d2607324e0 msgr2=0x55d260ade070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 shutdown_connections 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 1 -- 172.21.15.19:0/1355641267 wait complete. 2024-04-25T10:54:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.296+0000 7f950eb79740 20 client.15405 trim_cache size 0 max 16384 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: "a8699705-fabe-4f32-a5b0-3fe4f86e0e24": { 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "a8699705-fabe-4f32-a5b0-3fe4f86e0e24", 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:54:34.303 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f94ec0afc60 msgr2=0x7f94ec0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f94ec0afc60 0x7f94ec0b2170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94f8023000 tx=0x7f94f8021000 comp rx=0 tx=0).stop 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9500103c20 msgr2=0x7f9500199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9500103c20 0x7f9500199950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006010 tx=0x7f94f0004270 comp rx=0 tx=0).stop 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 shutdown_connections 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f94ec0afc60 0x7f94ec0b2170 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9500108920 0x7f95001a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9500103c20 0x7f9500199950 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 --2- 172.21.15.19:0/55302405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95001029e0 0x7f9500199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 >> 172.21.15.19:0/55302405 conn(0x7f95000fdbd0 msgr2=0x7f9500104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 shutdown_connections 2024-04-25T10:54:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.297+0000 7f9507bf3640 1 -- 172.21.15.19:0/55302405 wait complete. 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4167033340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: Unable to set osd_memory_target on smithi112 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3087221439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:54:34.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:33 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:54:34.311 INFO:tasks.cephfs.filesystem:scrub status for tag:a8699705-fabe-4f32-a5b0-3fe4f86e0e24 - {'path': '/', 'tag': 'a8699705-fabe-4f32-a5b0-3fe4f86e0e24', 'options': 'recursive,force'} 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4167033340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: Unable to set osd_memory_target on smithi112 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.312 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3087221439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:54:34.313 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:33 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:54:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.397+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f277002db50 con 0x7f2770104010 2024-04-25T10:54:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.402+0000 7f27677fe640 1 -- 172.21.15.19:0/2893572214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f276c0cbfb0 con 0x7f2770104010 2024-04-25T10:54:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.404+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f275c0afc50 msgr2=0x7f275c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.404+0000 7f27782af640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f275c0afc50 0x7f275c0b2160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f277019d4e0 tx=0x7f276000b390 comp rx=0 tx=0).stop 2024-04-25T10:54:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.404+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770104010 msgr2=0x7f277019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.404+0000 7f27782af640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770104010 0x7f277019c440 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f276c006010 tx=0x7f276c004270 comp rx=0 tx=0).stop 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 shutdown_connections 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f275c0afc50 0x7f275c0b2160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010ba20 0x7f27701a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2770108ef0 0x7f277019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 --2- 172.21.15.19:0/2893572214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770104010 0x7f277019c440 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 >> 172.21.15.19:0/2893572214 conn(0x7f27700fd390 msgr2=0x7f277010af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 shutdown_connections 2024-04-25T10:54:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.405+0000 7f27782af640 1 -- 172.21.15.19:0/2893572214 wait complete. 2024-04-25T10:54:34.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T10:54:34.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.536+0000 7f1200eb6640 1 -- 172.21.15.19:0/4209652835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc104c60 msgr2=0x7f11fc105070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.536+0000 7f1200eb6640 1 --2- 172.21.15.19:0/4209652835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc104c60 0x7f11fc105070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009860 tx=0x7f11ec02a340 comp rx=0 tx=0).stop 2024-04-25T10:54:34.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 -- 172.21.15.19:0/4209652835 shutdown_connections 2024-04-25T10:54:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 --2- 172.21.15.19:0/4209652835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10e0b0 0x7f11fc1105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 --2- 172.21.15.19:0/4209652835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11fc10b6c0 0x7f11fc10dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 --2- 172.21.15.19:0/4209652835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc104c60 0x7f11fc105070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 -- 172.21.15.19:0/4209652835 >> 172.21.15.19:0/4209652835 conn(0x7f11fc0fe260 msgr2=0x7f11fc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 -- 172.21.15.19:0/4209652835 shutdown_connections 2024-04-25T10:54:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 -- 172.21.15.19:0/4209652835 wait complete. 2024-04-25T10:54:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 Processor -- start 2024-04-25T10:54:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.537+0000 7f1200eb6640 1 -- start start 2024-04-25T10:54:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f1200eb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11fc104c60 0x7f11fc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f1200eb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10b6c0 0x7f11fc19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f1200eb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc10e0b0 0x7f11fc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f1200eb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc113d10 con 0x7f11fc10b6c0 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f1200eb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc113b90 con 0x7f11fc104c60 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f1200eb6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc113e90 con 0x7f11fc10e0b0 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10b6c0 0x7f11fc19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11fc104c60 0x7f11fc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10b6c0 0x7f11fc19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49136/0 (socket says 172.21.15.19:49136) 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11f9d9b640 1 -- 172.21.15.19:0/827490340 learned_addr learned my addr 172.21.15.19:0/827490340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fad9d640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc10e0b0 0x7f11fc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fa59c640 1 -- 172.21.15.19:0/827490340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc10e0b0 msgr2=0x7f11fc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fa59c640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc10e0b0 0x7f11fc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fa59c640 1 -- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10b6c0 msgr2=0x7f11fc19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fa59c640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10b6c0 0x7f11fc19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fa59c640 1 -- 172.21.15.19:0/827490340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc19f570 con 0x7f11fc104c60 2024-04-25T10:54:34.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11f9d9b640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10b6c0 0x7f11fc19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:54:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.538+0000 7f11fa59c640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11fc104c60 0x7f11fc19a710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11ec035040 tx=0x7f11ec032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.539+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec040020 con 0x7f11fc104c60 2024-04-25T10:54:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.539+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc19f800 con 0x7f11fc104c60 2024-04-25T10:54:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.539+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec030e10 con 0x7f11fc104c60 2024-04-25T10:54:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.539+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec037040 con 0x7f11fc104c60 2024-04-25T10:54:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.539+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc19fcf0 con 0x7f11fc104c60 2024-04-25T10:54:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.540+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11fc050740 con 0x7f11fc104c60 2024-04-25T10:54:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.541+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f11ec043020 con 0x7f11fc104c60 2024-04-25T10:54:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.542+0000 7f11eb7fe640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f11e40afc50 0x7f11e40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.542+0000 7f11f9d9b640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f11e40afc50 0x7f11e40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.542+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f11ec0fee60 con 0x7f11fc104c60 2024-04-25T10:54:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.542+0000 7f11f9d9b640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f11e40afc50 0x7f11e40b2160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11f00095f0 tx=0x7f11f0005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.545+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11ec04a020 con 0x7f11fc104c60 2024-04-25T10:54:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.630+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7f11ec0c33f0 con 0x7f11fc104c60 2024-04-25T10:54:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.785+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f11fc1a7240 con 0x7f11fc104c60 2024-04-25T10:54:34.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.804+0000 7f11eb7fe640 1 -- 172.21.15.19:0/827490340 <== mon.2 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f11ec0c7860 con 0x7f11fc104c60 2024-04-25T10:54:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f11e40afc50 msgr2=0x7f11e40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f11e40afc50 0x7f11e40b2160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11f00095f0 tx=0x7f11f0005f90 comp rx=0 tx=0).stop 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11fc104c60 msgr2=0x7f11fc19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11fc104c60 0x7f11fc19a710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11ec035040 tx=0x7f11ec032d50 comp rx=0 tx=0).stop 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 shutdown_connections 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f11e40afc50 0x7f11e40b2160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc10e0b0 0x7f11fc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc10b6c0 0x7f11fc19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 --2- 172.21.15.19:0/827490340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11fc104c60 0x7f11fc19a710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.806+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 >> 172.21.15.19:0/827490340 conn(0x7f11fc0fe260 msgr2=0x7f11fc10e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.807+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 shutdown_connections 2024-04-25T10:54:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.807+0000 7f1200eb6640 1 -- 172.21.15.19:0/827490340 wait complete. 2024-04-25T10:54:34.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T10:54:34.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.943+0000 7fc4112bd640 1 -- 172.21.15.19:0/1804341723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108f20 msgr2=0x7fc40c102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.943+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1804341723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108f20 0x7fc40c102b60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a3f0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.943+0000 7fc4112bd640 1 -- 172.21.15.19:0/1804341723 shutdown_connections 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.943+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1804341723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc40c1031d0 0x7fc40c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.943+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1804341723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108f20 0x7fc40c102b60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.943+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1804341723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc40c108a80 0x7fc40c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.943+0000 7fc4112bd640 1 -- 172.21.15.19:0/1804341723 >> 172.21.15.19:0/1804341723 conn(0x7fc40c079170 msgr2=0x7fc40c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.944+0000 7fc4112bd640 1 -- 172.21.15.19:0/1804341723 shutdown_connections 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.944+0000 7fc4112bd640 1 -- 172.21.15.19:0/1804341723 wait complete. 2024-04-25T10:54:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.944+0000 7fc4112bd640 1 Processor -- start 2024-04-25T10:54:34.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.944+0000 7fc4112bd640 1 -- start start 2024-04-25T10:54:34.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc40c1031d0 0x7fc40c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108a80 0x7fc40c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc40c108f20 0x7fc40c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112c70 con 0x7fc40c108a80 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112af0 con 0x7fc40c1031d0 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112df0 con 0x7fc40c108f20 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108a80 0x7fc40c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108a80 0x7fc40c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49140/0 (socket says 172.21.15.19:49140) 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 -- 172.21.15.19:0/1386625579 learned_addr learned my addr 172.21.15.19:0/1386625579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40affd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc40c1031d0 0x7fc40c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 -- 172.21.15.19:0/1386625579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc40c108f20 msgr2=0x7fc40c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc40c108f20 0x7fc40c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 -- 172.21.15.19:0/1386625579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc40c1031d0 msgr2=0x7fc40c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc40c1031d0 0x7fc40c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 -- 172.21.15.19:0/1386625579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c19e430 con 0x7fc40c108a80 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc40a7fc640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108a80 0x7fc40c199bb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc400031a90 tx=0x7fc400004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc3f3fff640 1 -- 172.21.15.19:0/1386625579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40003f030 con 0x7fc40c108a80 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc3f3fff640 1 -- 172.21.15.19:0/1386625579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400003cb0 con 0x7fc40c108a80 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c19e6c0 con 0x7fc40c108a80 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc3f3fff640 1 -- 172.21.15.19:0/1386625579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400035070 con 0x7fc40c108a80 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.945+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c19ebb0 con 0x7fc40c108a80 2024-04-25T10:54:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.946+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c050740 con 0x7fc40c108a80 2024-04-25T10:54:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.948+0000 7fc3f3fff640 1 -- 172.21.15.19:0/1386625579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7fc400042030 con 0x7fc40c108a80 2024-04-25T10:54:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.948+0000 7fc3f3fff640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc3f40afce0 0x7fc3f40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.949+0000 7fc40affd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc3f40afce0 0x7fc3f40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.949+0000 7fc3f3fff640 1 -- 172.21.15.19:0/1386625579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc4000ffc60 con 0x7fc40c108a80 2024-04-25T10:54:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.949+0000 7fc40affd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc3f40afce0 0x7fc3f40b21f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc40c103760 tx=0x7fc3fc00c4a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:34.952+0000 7fc3f3fff640 1 -- 172.21.15.19:0/1386625579 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4000c83c0 con 0x7fc40c108a80 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Unable to set osd_memory_target on smithi019 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2248855806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2893572214' entity='client.admin' 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='client.? ' entity='client.admin' 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:34 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.191+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc40c101810 con 0x7fc3f40afce0 2024-04-25T10:54:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.225+0000 7fc3f3fff640 1 -- 172.21.15.19:0/1386625579 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc40c101810 con 0x7fc3f40afce0 2024-04-25T10:54:35.226 INFO:teuthology.orchestra.run.smithi019.stdout:Schedule set for path / 2024-04-25T10:54:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc3f40afce0 msgr2=0x7fc3f40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc3f40afce0 0x7fc3f40b21f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc40c103760 tx=0x7fc3fc00c4a0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108a80 msgr2=0x7fc40c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108a80 0x7fc40c199bb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc400031a90 tx=0x7fc400004270 comp rx=0 tx=0).stop 2024-04-25T10:54:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 shutdown_connections 2024-04-25T10:54:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc3f40afce0 0x7fc3f40b21f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc40c108f20 0x7fc40c19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc40c108a80 0x7fc40c199bb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 --2- 172.21.15.19:0/1386625579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc40c1031d0 0x7fc40c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 >> 172.21.15.19:0/1386625579 conn(0x7fc40c079170 msgr2=0x7fc40c101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 shutdown_connections 2024-04-25T10:54:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.227+0000 7fc4112bd640 1 -- 172.21.15.19:0/1386625579 wait complete. 2024-04-25T10:54:35.239 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T10:54:35.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T10:54:35.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Unable to set osd_memory_target on smithi019 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T10:54:35.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T10:54:35.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:54:35.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:54:35.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2248855806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2893572214' entity='client.admin' 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='client.? ' entity='client.admin' 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.253 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:34 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Unable to set osd_memory_target on smithi019 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:35.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2248855806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2893572214' entity='client.admin' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='client.? ' entity='client.admin' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:34 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:35.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.354+0000 7f4134da8640 1 -- 172.21.15.19:0/1924754430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 msgr2=0x7f4130103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.354+0000 7f4134da8640 1 --2- 172.21.15.19:0/1924754430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f4130103c50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4124009900 tx=0x7f412402a410 comp rx=0 tx=0).stop 2024-04-25T10:54:35.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 -- 172.21.15.19:0/1924754430 shutdown_connections 2024-04-25T10:54:35.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 --2- 172.21.15.19:0/1924754430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41301042c0 0x7f4130110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 --2- 172.21.15.19:0/1924754430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f4130103c50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 --2- 172.21.15.19:0/1924754430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4130109b70 0x7f4130103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 -- 172.21.15.19:0/1924754430 >> 172.21.15.19:0/1924754430 conn(0x7f41300ff210 msgr2=0x7f4130101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:35.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 -- 172.21.15.19:0/1924754430 shutdown_connections 2024-04-25T10:54:35.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 -- 172.21.15.19:0/1924754430 wait complete. 2024-04-25T10:54:35.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 Processor -- start 2024-04-25T10:54:35.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.355+0000 7f4134da8640 1 -- start start 2024-04-25T10:54:35.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f4134da8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41301042c0 0x7f413019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f4134da8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4130109b70 0x7f413019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f4134da8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f413019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f4134da8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4130113d60 con 0x7f413010a010 2024-04-25T10:54:35.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f4134da8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4130113be0 con 0x7f4130109b70 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f4134da8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4130113ee0 con 0x7f41301042c0 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f413019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f413019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49154/0 (socket says 172.21.15.19:49154) 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 -- 172.21.15.19:0/1964537335 learned_addr learned my addr 172.21.15.19:0/1964537335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412e59c640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41301042c0 0x7f413019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412dd9b640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4130109b70 0x7f413019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 -- 172.21.15.19:0/1964537335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41301042c0 msgr2=0x7f413019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41301042c0 0x7f413019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 -- 172.21.15.19:0/1964537335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4130109b70 msgr2=0x7f413019ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4130109b70 0x7f413019ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 -- 172.21.15.19:0/1964537335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413019f620 con 0x7f413010a010 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f412ed9d640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f413019efe0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f412800eb10 tx=0x7f412800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.356+0000 7f411f7fe640 1 -- 172.21.15.19:0/1964537335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128004b20 con 0x7f413010a010 2024-04-25T10:54:35.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.357+0000 7f411f7fe640 1 -- 172.21.15.19:0/1964537335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4128004cc0 con 0x7f413010a010 2024-04-25T10:54:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.357+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413019f910 con 0x7f413010a010 2024-04-25T10:54:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.357+0000 7f411f7fe640 1 -- 172.21.15.19:0/1964537335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128010670 con 0x7f413010a010 2024-04-25T10:54:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.357+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41301a7290 con 0x7f413010a010 2024-04-25T10:54:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.358+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4130050740 con 0x7f413010a010 2024-04-25T10:54:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.359+0000 7f411f7fe640 1 -- 172.21.15.19:0/1964537335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7f4128005000 con 0x7f413010a010 2024-04-25T10:54:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.360+0000 7f411f7fe640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41180afce0 0x7f41180b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.360+0000 7f412e59c640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41180afce0 0x7f41180b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.360+0000 7f411f7fe640 1 -- 172.21.15.19:0/1964537335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f41280dfa50 con 0x7f413010a010 2024-04-25T10:54:35.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.360+0000 7f412e59c640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41180afce0 0x7f41180b21f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4120005e00 tx=0x7f4120005c50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:35.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.364+0000 7f411f7fe640 1 -- 172.21.15.19:0/1964537335 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4128053ff0 con 0x7f413010a010 2024-04-25T10:54:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.577+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f413010ff30 con 0x7f41180afce0 2024-04-25T10:54:35.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.603+0000 7f411f7fe640 1 -- 172.21.15.19:0/1964537335 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f413010ff30 con 0x7f41180afce0 2024-04-25T10:54:35.604 INFO:teuthology.orchestra.run.smithi019.stdout:Retention added to path / 2024-04-25T10:54:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41180afce0 msgr2=0x7f41180b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41180afce0 0x7f41180b21f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4120005e00 tx=0x7f4120005c50 comp rx=0 tx=0).stop 2024-04-25T10:54:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 msgr2=0x7f413019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f413019efe0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f412800eb10 tx=0x7f412800c2d0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 shutdown_connections 2024-04-25T10:54:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41180afce0 0x7f41180b21f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f413010a010 0x7f413019efe0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4130109b70 0x7f413019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 --2- 172.21.15.19:0/1964537335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41301042c0 0x7f413019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 >> 172.21.15.19:0/1964537335 conn(0x7f41300ff210 msgr2=0x7f413010f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 shutdown_connections 2024-04-25T10:54:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.605+0000 7f4134da8640 1 -- 172.21.15.19:0/1964537335 wait complete. 2024-04-25T10:54:35.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 -- 172.21.15.19:0/4294697664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 msgr2=0x7f17fc10f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 --2- 172.21.15.19:0/4294697664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc10f4f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009900 tx=0x7f17f402a410 comp rx=0 tx=0).stop 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 -- 172.21.15.19:0/4294697664 shutdown_connections 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 --2- 172.21.15.19:0/4294697664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc10f4f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 --2- 172.21.15.19:0/4294697664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17fc108f00 0x7f17fc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 --2- 172.21.15.19:0/4294697664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17fc108a60 0x7f17fc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 -- 172.21.15.19:0/4294697664 >> 172.21.15.19:0/4294697664 conn(0x7f17fc0781b0 msgr2=0x7f17fc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 -- 172.21.15.19:0/4294697664 shutdown_connections 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 -- 172.21.15.19:0/4294697664 wait complete. 2024-04-25T10:54:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.729+0000 7f1801aa7640 1 Processor -- start 2024-04-25T10:54:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f1801aa7640 1 -- start start 2024-04-25T10:54:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f1801aa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f1801aa7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17fc108a60 0x7f17fc19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f1801aa7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17fc108f00 0x7f17fc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f1801aa7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112ca0 con 0x7f17fc1031c0 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f1801aa7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112b20 con 0x7f17fc108f00 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f1801aa7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112e20 con 0x7f17fc108a60 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49168/0 (socket says 172.21.15.19:49168) 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 -- 172.21.15.19:0/1820506285 learned_addr learned my addr 172.21.15.19:0/1820506285 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fbfff640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17fc108f00 0x7f17fc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17faffd640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17fc108a60 0x7f17fc19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 -- 172.21.15.19:0/1820506285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17fc108a60 msgr2=0x7f17fc19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17fc108a60 0x7f17fc19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 -- 172.21.15.19:0/1820506285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17fc108f00 msgr2=0x7f17fc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17fc108f00 0x7f17fc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.730+0000 7f17fb7fe640 1 -- 172.21.15.19:0/1820506285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc19f510 con 0x7f17fc1031c0 2024-04-25T10:54:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.731+0000 7f17fb7fe640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc19a730 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00cd10 tx=0x7f17ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.731+0000 7f17f8ff9640 1 -- 172.21.15.19:0/1820506285 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00f450 con 0x7f17fc1031c0 2024-04-25T10:54:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.731+0000 7f17f8ff9640 1 -- 172.21.15.19:0/1820506285 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17ec00faa0 con 0x7f17fc1031c0 2024-04-25T10:54:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.731+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc19f800 con 0x7f17fc1031c0 2024-04-25T10:54:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.731+0000 7f17f8ff9640 1 -- 172.21.15.19:0/1820506285 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec018c10 con 0x7f17fc1031c0 2024-04-25T10:54:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.731+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a7200 con 0x7f17fc1031c0 2024-04-25T10:54:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.732+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17fc06a110 con 0x7f17fc1031c0 2024-04-25T10:54:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.733+0000 7f17f8ff9640 1 -- 172.21.15.19:0/1820506285 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7f17ec00fc50 con 0x7f17fc1031c0 2024-04-25T10:54:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.734+0000 7f17f8ff9640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f17e80afd60 0x7f17e80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.734+0000 7f17faffd640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f17e80afd60 0x7f17e80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.734+0000 7f17f8ff9640 1 -- 172.21.15.19:0/1820506285 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f17ec0df980 con 0x7f17fc1031c0 2024-04-25T10:54:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.734+0000 7f17faffd640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f17e80afd60 0x7f17e80b2270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17fc100280 tx=0x7f17e4005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.737+0000 7f17f8ff9640 1 -- 172.21.15.19:0/1820506285 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17ec014030 con 0x7f17fc1031c0 2024-04-25T10:54:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.948+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f17fc106350 con 0x7f17e80afd60 2024-04-25T10:54:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.956+0000 7f17f8ff9640 1 -- 172.21.15.19:0/1820506285 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f17fc106350 con 0x7f17e80afd60 2024-04-25T10:54:35.957 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T10:54:35", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f17e80afd60 msgr2=0x7f17e80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f17e80afd60 0x7f17e80b2270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17fc100280 tx=0x7f17e4005f90 comp rx=0 tx=0).stop 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 msgr2=0x7f17fc19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc19a730 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00cd10 tx=0x7f17ec00a620 comp rx=0 tx=0).stop 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 shutdown_connections 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f17e80afd60 0x7f17e80b2270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17fc108f00 0x7f17fc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17fc108a60 0x7f17fc19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 --2- 172.21.15.19:0/1820506285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc1031c0 0x7f17fc19a730 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 >> 172.21.15.19:0/1820506285 conn(0x7f17fc0781b0 msgr2=0x7f17fc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 shutdown_connections 2024-04-25T10:54:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:35.958+0000 7f1801aa7640 1 -- 172.21.15.19:0/1820506285 wait complete. 2024-04-25T10:54:35.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 -- 172.21.15.19:0/3547679112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084108a60 msgr2=0x7fc084102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3547679112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084108a60 0x7fc084102610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc080009900 tx=0x7fc08002a410 comp rx=0 tx=0).stop 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 -- 172.21.15.19:0/3547679112 shutdown_connections 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3547679112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0841031c0 0x7fc08410f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3547679112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc084108f00 0x7fc084102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3547679112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc084108a60 0x7fc084102610 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 -- 172.21.15.19:0/3547679112 >> 172.21.15.19:0/3547679112 conn(0x7fc0840781b0 msgr2=0x7fc0840785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 -- 172.21.15.19:0/3547679112 shutdown_connections 2024-04-25T10:54:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.084+0000 7fc08be6e640 1 -- 172.21.15.19:0/3547679112 wait complete. 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 Processor -- start 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 -- start start 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0841031c0 0x7fc08419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc084108a60 0x7fc08419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc084108f00 0x7fc08419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc084112c60 con 0x7fc0841031c0 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc084112ae0 con 0x7fc084108a60 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08be6e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc084112de0 con 0x7fc084108f00 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0841031c0 0x7fc08419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0841031c0 0x7fc08419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49186/0 (socket says 172.21.15.19:49186) 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 -- 172.21.15.19:0/3102959437 learned_addr learned my addr 172.21.15.19:0/3102959437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089409640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc084108a60 0x7fc08419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc08a40b640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc084108f00 0x7fc08419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 -- 172.21.15.19:0/3102959437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc084108f00 msgr2=0x7fc08419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc084108f00 0x7fc08419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 -- 172.21.15.19:0/3102959437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc084108a60 msgr2=0x7fc08419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc084108a60 0x7fc08419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.085+0000 7fc089c0a640 1 -- 172.21.15.19:0/3102959437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08419f570 con 0x7fc0841031c0 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.086+0000 7fc089c0a640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0841031c0 0x7fc08419a790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc080031a90 tx=0x7fc080004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.086+0000 7fc07affd640 1 -- 172.21.15.19:0/3102959437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08003f030 con 0x7fc0841031c0 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.086+0000 7fc07affd640 1 -- 172.21.15.19:0/3102959437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc080003cb0 con 0x7fc0841031c0 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.086+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08419f800 con 0x7fc0841031c0 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.086+0000 7fc07affd640 1 -- 172.21.15.19:0/3102959437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080035070 con 0x7fc0841031c0 2024-04-25T10:54:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.086+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08419fcf0 con 0x7fc0841031c0 2024-04-25T10:54:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.087+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08406a110 con 0x7fc0841031c0 2024-04-25T10:54:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.088+0000 7fc07affd640 1 -- 172.21.15.19:0/3102959437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7fc080042030 con 0x7fc0841031c0 2024-04-25T10:54:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.089+0000 7fc07affd640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc0700afce0 0x7fc0700b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.089+0000 7fc089409640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc0700afce0 0x7fc0700b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.089+0000 7fc07affd640 1 -- 172.21.15.19:0/3102959437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc0800ff7a0 con 0x7fc0841031c0 2024-04-25T10:54:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.089+0000 7fc089409640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc0700afce0 0x7fc0700b21f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc08410f7d0 tx=0x7fc074005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.094+0000 7fc07affd640 1 -- 172.21.15.19:0/3102959437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc080003e60 con 0x7fc0841031c0 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:54:36.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:54:36.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:54:36.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:36.252 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:35 smithi169 ceph-mon[33049]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.307+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fc08406aad0 con 0x7fc0700afce0 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:36.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:35 smithi112 ceph-mon[32799]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:36.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.314+0000 7fc07affd640 1 -- 172.21.15.19:0/3102959437 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fc08406aad0 con 0x7fc0700afce0 2024-04-25T10:54:36.315 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc0700afce0 msgr2=0x7fc0700b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc0700afce0 0x7fc0700b21f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc08410f7d0 tx=0x7fc074005f90 comp rx=0 tx=0).stop 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0841031c0 msgr2=0x7fc08419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0841031c0 0x7fc08419a790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc080031a90 tx=0x7fc080004270 comp rx=0 tx=0).stop 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 shutdown_connections 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc0700afce0 0x7fc0700b21f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc084108f00 0x7fc08419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc084108a60 0x7fc08419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 --2- 172.21.15.19:0/3102959437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0841031c0 0x7fc08419a790 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 >> 172.21.15.19:0/3102959437 conn(0x7fc0840781b0 msgr2=0x7fc08406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:36.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 shutdown_connections 2024-04-25T10:54:36.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:54:36.316+0000 7fc08be6e640 1 -- 172.21.15.19:0/3102959437 wait complete. 2024-04-25T10:54:36.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:54:36.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:54:36.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: Updating smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: Updating smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: Updating smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:54:36.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:35 smithi019 ceph-mon[29209]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:36.329 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T10:54:36.354 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T10:54:36.365 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T10:54:36.383 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:54:36.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T10:54:36.397 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T10:54:36.409 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:54:36.410 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T10:54:36.410 INFO:tasks.workunit:timeout=3h 2024-04-25T10:54:36.410 INFO:tasks.workunit:cleanup=True 2024-04-25T10:54:36.410 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:36.426 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:36.426 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T10:54:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T10:54:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:54:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T10:54:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 10:52:38.241988967 +0000 2024-04-25T10:54:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 10:52:38.241988967 +0000 2024-04-25T10:54:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 10:54:26.813444630 +0000 2024-04-25T10:54:36.427 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 10:52:38.241988967 +0000 2024-04-25T10:54:36.427 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T10:54:36.427 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T10:54:36.496 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:54:36.552 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T10:54:37.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:37 smithi112 ceph-mon[32799]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:37 smithi112 ceph-mon[32799]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:37.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:37 smithi112 ceph-mon[32799]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:37 smithi112 ceph-mon[32799]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:37 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:54:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:37 smithi019 ceph-mon[29209]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:37 smithi019 ceph-mon[29209]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:37 smithi019 ceph-mon[29209]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:37 smithi019 ceph-mon[29209]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:37 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:54:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:37 smithi169 ceph-mon[33049]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:37 smithi169 ceph-mon[33049]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:54:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:37 smithi169 ceph-mon[33049]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:37 smithi169 ceph-mon[33049]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:54:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:37 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:54:38.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:38 smithi169 ceph-mon[33049]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 131 KiB/s wr, 28 op/s 2024-04-25T10:54:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:38 smithi112 ceph-mon[32799]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 131 KiB/s wr, 28 op/s 2024-04-25T10:54:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:38 smithi019 ceph-mon[29209]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 131 KiB/s wr, 28 op/s 2024-04-25T10:54:41.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:40 smithi169 ceph-mon[33049]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 21 op/s 2024-04-25T10:54:41.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:40 smithi112 ceph-mon[32799]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 21 op/s 2024-04-25T10:54:41.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:40 smithi019 ceph-mon[29209]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 21 op/s 2024-04-25T10:54:43.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:42 smithi169 ceph-mon[33049]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 21 op/s 2024-04-25T10:54:43.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:42 smithi112 ceph-mon[32799]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 21 op/s 2024-04-25T10:54:43.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:42 smithi019 ceph-mon[29209]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 21 op/s 2024-04-25T10:54:45.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:44 smithi169 ceph-mon[33049]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:45.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:44 smithi112 ceph-mon[32799]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:45.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:44 smithi019 ceph-mon[29209]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:47.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:46 smithi169 ceph-mon[33049]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:47.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:46 smithi112 ceph-mon[32799]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:46 smithi019 ceph-mon[29209]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:49.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:48 smithi169 ceph-mon[33049]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:49.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:48 smithi112 ceph-mon[32799]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:48 smithi019 ceph-mon[29209]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:54:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 -- 172.21.15.19:0/3226433641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108a60 msgr2=0x7ff3fc102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 --2- 172.21.15.19:0/3226433641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108a60 0x7ff3fc102610 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009900 tx=0x7ff3f402a410 comp rx=0 tx=0).stop 2024-04-25T10:54:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 -- 172.21.15.19:0/3226433641 shutdown_connections 2024-04-25T10:54:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 --2- 172.21.15.19:0/3226433641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc1031c0 0x7ff3fc10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 --2- 172.21.15.19:0/3226433641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc108f00 0x7ff3fc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 --2- 172.21.15.19:0/3226433641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108a60 0x7ff3fc102610 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 -- 172.21.15.19:0/3226433641 >> 172.21.15.19:0/3226433641 conn(0x7ff3fc0781b0 msgr2=0x7ff3fc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 -- 172.21.15.19:0/3226433641 shutdown_connections 2024-04-25T10:54:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.550+0000 7ff400ab1640 1 -- 172.21.15.19:0/3226433641 wait complete. 2024-04-25T10:54:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 Processor -- start 2024-04-25T10:54:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 -- start start 2024-04-25T10:54:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc1031c0 0x7ff3fc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc108a60 0x7ff3fc19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108f00 0x7ff3fc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc113d80 con 0x7ff3fc108f00 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc113c00 con 0x7ff3fc1031c0 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.551+0000 7ff400ab1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc113f00 con 0x7ff3fc108a60 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108f00 0x7ff3fc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108f00 0x7ff3fc19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56980/0 (socket says 172.21.15.19:56980) 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 -- 172.21.15.19:0/2872234737 learned_addr learned my addr 172.21.15.19:0/2872234737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc108a60 0x7ff3fc19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc1031c0 0x7ff3fc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 -- 172.21.15.19:0/2872234737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc108a60 msgr2=0x7ff3fc19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc108a60 0x7ff3fc19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 -- 172.21.15.19:0/2872234737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc1031c0 msgr2=0x7ff3fc19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc1031c0 0x7ff3fc19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 -- 172.21.15.19:0/2872234737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc19f5c0 con 0x7ff3fc108f00 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3fad9d640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108f00 0x7ff3fc19ef80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00eb10 tx=0x7ff3ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3eb7fe640 1 -- 172.21.15.19:0/2872234737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec004b20 con 0x7ff3fc108f00 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc19f8b0 con 0x7ff3fc108f00 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3eb7fe640 1 -- 172.21.15.19:0/2872234737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3ec004cc0 con 0x7ff3fc108f00 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff3eb7fe640 1 -- 172.21.15.19:0/2872234737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec010670 con 0x7ff3fc108f00 2024-04-25T10:54:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.552+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc19fdb0 con 0x7ff3fc108f00 2024-04-25T10:54:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.555+0000 7ff407a3a740 1 Processor -- start 2024-04-25T10:54:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.555+0000 7ff3eb7fe640 1 -- 172.21.15.19:0/2872234737 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7ff3ec005020 con 0x7ff3fc108f00 2024-04-25T10:54:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.555+0000 7ff407a3a740 1 -- start start 2024-04-25T10:54:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6581b0 0x5602af9f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602af9f8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 0x5602af9f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff407a3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5602af95e460 con 0x5602af6581b0 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff407a3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5602af918590 con 0x5602af62f2f0 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff407a3a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5602af95a420 con 0x5602af6374d0 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6581b0 0x5602af9f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6581b0 0x5602af9f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56994/0 (socket says 172.21.15.19:56994) 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/803475174 learned_addr learned my addr 172.21.15.19:0/803475174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602af9f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3eb7fe640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff3e40afce0 0x7ff3e40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/803475174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 msgr2=0x5602af9f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 0x5602af9f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/803475174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 msgr2=0x5602af9f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602af9f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/803475174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602af9250d0 con 0x5602af6581b0 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602af9f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3fad9d640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff3e40afce0 0x7ff3e40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3eb7fe640 1 -- 172.21.15.19:0/2872234737 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff3ec0df810 con 0x7ff3fc108f00 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.556+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6581b0 0x5602af9f7af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0004650 tx=0x7ff3f000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14ad89e6c7e2a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.557+0000 7ff3e8ff9640 1 -- 172.21.15.19:0/803475174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003bd0 con 0x5602af6581b0 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.557+0000 7ff3e8ff9640 1 -- 172.21.15.19:0/803475174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f0016030 con 0x5602af6581b0 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.557+0000 7ff3e8ff9640 1 -- 172.21.15.19:0/803475174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f000f620 con 0x5602af6581b0 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 -- 172.21.15.19:0/803475174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6581b0 msgr2=0x5602af9f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff3fad9d640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff3e40afce0 0x7ff3e40b21f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0dcf30 tx=0x7ff3ec076370 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6581b0 0x5602af9f7af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0004650 tx=0x7ff3f000bbe0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 -- 172.21.15.19:0/803475174 shutdown_connections 2024-04-25T10:54:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 0x5602af9f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602af9f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 --2- 172.21.15.19:0/803475174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6581b0 0x5602af9f7af0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 -- 172.21.15.19:0/803475174 >> 172.21.15.19:0/803475174 conn(0x5602af64c300 msgr2=0x5602af9f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 -- 172.21.15.19:0/803475174 shutdown_connections 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 -- 172.21.15.19:0/803475174 wait complete. 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 -- start start 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.558+0000 7ff407a3a740 1 Processor -- start 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa03b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 0x5602afa04040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6581b0 0x5602afa01ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5602afa04bc0 con 0x5602af62f2f0 2024-04-25T10:54:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5602af8e5b60 con 0x5602af6581b0 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5602afa04d40 con 0x5602af6374d0 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 10 client.0 _ll_register_callbacks cb 0x5602af90a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa03b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa03b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57000/0 (socket says 172.21.15.19:57000) 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/4086900175 learned_addr learned my addr 172.21.15.19:0/4086900175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/4086900175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 msgr2=0x5602afa04040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 0x5602afa04040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/4086900175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6581b0 msgr2=0x5602afa01ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6581b0 0x5602afa01ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/4086900175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602af670210 con 0x5602af62f2f0 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa03b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006010 tx=0x7ff3f0003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4988c22e40301d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/4086900175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0005220 con 0x5602af62f2f0 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/4086900175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff3f00053c0 con 0x5602af62f2f0 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.559+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f0016030 con 0x5602af62f2f0 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/4086900175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff3f0010430 con 0x5602af62f2f0 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 10 client.15447 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5602af73c620 con 0x5602af62f2f0 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/4086900175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f0016030 con 0x5602af62f2f0 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/4086900175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff3f0023030 con 0x5602af62f2f0 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 10 client.15447 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 20 client.15447 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af616d80 0x5602afa146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 4 client.15447 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:54:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.560+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602af96c070 con 0x5602af616d80 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.561+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af616d80 0x5602afa146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.561+0000 7ff3e1ffb640 10 client.15447 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.561+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af616d80 0x5602afa146c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8894425f58d42419 server_cookie=a3b3cd32d3fcdd34 in_seq=0 out_seq=0 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/4086900175 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5602af96c070 con 0x5602af616d80 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff3e1ffb640 10 client.15447 handle_command_reply: tid=0 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 client.15447 shutdown 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 msgr2=0x5602afa03b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa03b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006010 tx=0x7ff3f0003bd0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 shutdown_connections 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af616d80 0x5602afa146c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6581b0 0x5602afa01ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6374d0 0x5602afa04040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 --2- 172.21.15.19:0/4086900175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa03b00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff3e1ffb640 0 client.15447 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 >> 172.21.15.19:0/4086900175 conn(0x5602af64c300 msgr2=0x5602af9f7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 shutdown_connections 2024-04-25T10:54:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 1 -- 172.21.15.19:0/4086900175 wait complete. 2024-04-25T10:54:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.562+0000 7ff407a3a740 20 client.15447 trim_cache size 0 max 16384 2024-04-25T10:54:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 Processor -- start 2024-04-25T10:54:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 -- start start 2024-04-25T10:54:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602afa00f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6374d0 0x5602af9f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa19d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1ad940 con 0x5602af6374d0 2024-04-25T10:54:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1b01c0 con 0x5602af62f2f0 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff407a3a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1adfb0 con 0x5602af6581b0 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6374d0 0x5602af9f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6374d0 0x5602af9f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57014/0 (socket says 172.21.15.19:57014) 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 -- 172.21.15.19:0/1046665046 learned_addr learned my addr 172.21.15.19:0/1046665046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602afa00f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa19d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 -- 172.21.15.19:0/1046665046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 msgr2=0x5602afa19d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa19d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 -- 172.21.15.19:0/1046665046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 msgr2=0x5602afa00f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602afa00f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3fa59c640 1 -- 172.21.15.19:0/1046665046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602afa02e90 con 0x5602af6374d0 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.578+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa19d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:54:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.579+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6374d0 0x5602af9f7f90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005f70 tx=0x7ff3f40095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f43d4b99f78c89e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.579+0000 7ff3e3fff640 1 -- 172.21.15.19:0/1046665046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4048070 con 0x5602af6374d0 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.579+0000 7ff3e3fff640 1 -- 172.21.15.19:0/1046665046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f4005040 con 0x5602af6374d0 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.579+0000 7ff3e3fff640 1 -- 172.21.15.19:0/1046665046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4035040 con 0x5602af6374d0 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- 172.21.15.19:0/1046665046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6374d0 msgr2=0x5602af9f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6374d0 0x5602af9f7f90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005f70 tx=0x7ff3f40095f0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- 172.21.15.19:0/1046665046 shutdown_connections 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa19d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af6374d0 0x5602af9f7f90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 --2- 172.21.15.19:0/1046665046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af62f2f0 0x5602afa00f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- 172.21.15.19:0/1046665046 >> 172.21.15.19:0/1046665046 conn(0x5602af64c300 msgr2=0x5602af9f7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- 172.21.15.19:0/1046665046 shutdown_connections 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- 172.21.15.19:0/1046665046 wait complete. 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- start start 2024-04-25T10:54:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 Processor -- start 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa73ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6374d0 0x5602afa741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1ae680 con 0x5602af62f2f0 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1afaf0 con 0x5602af6374d0 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1aed50 con 0x5602af6581b0 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.580+0000 7ff407a3a740 10 client.0 _ll_register_callbacks cb 0x7ff3f0004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa73ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa73ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57020/0 (socket says 172.21.15.19:57020) 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/2246897773 learned_addr learned my addr 172.21.15.19:0/2246897773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/2246897773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 msgr2=0x5602afa093d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/2246897773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6374d0 msgr2=0x5602afa741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6374d0 0x5602afa741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 -- 172.21.15.19:0/2246897773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602afa09ac0 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3f9d9b640 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa73ca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b470 tx=0x7ff3f0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d515801998cd64e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/2246897773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004040 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/2246897773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff3f00105d0 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f4005040 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/2246897773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff3f00107d0 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff407a3a740 10 client.15453 fetch_fsmap learned FSMap version 20 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.581+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5602afa7c270 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/2246897773 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f4005040 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/2246897773 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff3f0023040 con 0x5602af62f2f0 2024-04-25T10:54:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff407a3a740 10 client.15453 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff407a3a740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff407a3a740 20 client.15453 populate_metadata read hostname 'smithi019' 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff407a3a740 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af5ec1c0 0x5602afa7ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff407a3a740 4 client.15453 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602af965ee0 con 0x5602af5ec1c0 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.582+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af5ec1c0 0x5602afa7ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.583+0000 7ff3e1ffb640 10 client.15453 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.583+0000 7ff3fa59c640 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af5ec1c0 0x5602afa7ea10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f605d15397d00b21 server_cookie=aef0d3354db42e9a in_seq=0 out_seq=0 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.595+0000 7ff3e1ffb640 1 -- 172.21.15.19:0/2246897773 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602af965ee0 con 0x5602af5ec1c0 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.595+0000 7ff3e1ffb640 10 client.15453 handle_command_reply: tid=0 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.595+0000 7ff407a3a740 1 client.15453 shutdown 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 msgr2=0x5602afa73ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa73ca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b470 tx=0x7ff3f0003cd0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 shutdown_connections 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5602af5ec1c0 0x5602afa7ea10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602af6581b0 0x5602afa093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602af6374d0 0x5602afa741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff3e1ffb640 0 client.15453 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 --2- 172.21.15.19:0/2246897773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602af62f2f0 0x5602afa73ca0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 >> 172.21.15.19:0/2246897773 conn(0x5602af64c300 msgr2=0x5602afa01880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 shutdown_connections 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 1 -- 172.21.15.19:0/2246897773 wait complete. 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff407a3a740 20 client.15453 trim_cache size 0 max 16384 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:54:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:54:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff3e40afce0 msgr2=0x7ff3e40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff400ab1640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff3e40afce0 0x7ff3e40b21f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0dcf30 tx=0x7ff3ec076370 comp rx=0 tx=0).stop 2024-04-25T10:54:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108f00 msgr2=0x7ff3fc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.596+0000 7ff400ab1640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108f00 0x7ff3fc19ef80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00eb10 tx=0x7ff3ec00c2d0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 shutdown_connections 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff3e40afce0 0x7ff3e40b21f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc108f00 0x7ff3fc19ef80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc108a60 0x7ff3fc19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 --2- 172.21.15.19:0/2872234737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc1031c0 0x7ff3fc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 >> 172.21.15.19:0/2872234737 conn(0x7ff3fc0781b0 msgr2=0x7ff3fc104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 shutdown_connections 2024-04-25T10:54:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:54:49.597+0000 7ff400ab1640 1 -- 172.21.15.19:0/2872234737 wait complete. 2024-04-25T10:54:51.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:50 smithi169 ceph-mon[33049]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-25T10:54:51.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:50 smithi112 ceph-mon[32799]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-25T10:54:51.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:50 smithi019 ceph-mon[29209]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-25T10:54:53.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:53 smithi112 ceph-mon[32799]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-25T10:54:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:53 smithi019 ceph-mon[29209]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-25T10:54:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:53 smithi169 ceph-mon[33049]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-25T10:54:55.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:55 smithi112 ceph-mon[32799]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:55.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:55 smithi019 ceph-mon[29209]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:55.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:55 smithi169 ceph-mon[33049]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:57.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:57 smithi112 ceph-mon[32799]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:57.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:57 smithi019 ceph-mon[29209]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:57 smithi169 ceph-mon[33049]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:59.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:54:59 smithi112 ceph-mon[32799]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:59.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:54:59 smithi019 ceph-mon[29209]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:54:59.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:54:59 smithi169 ceph-mon[33049]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:55:01.016 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:55:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:55:00.616+0000 7f47fa008700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:55:01.016 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:55:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:55:00.616+0000 7f47fa008700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:55:01.016 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:55:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:55:00.616+0000 7f47fa008700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:55:01.016 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:55:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:55:00.616+0000 7f47fa008700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:55:01.016 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 10:55:00 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T10:55:00.616+0000 7f47fa008700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:55:01.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:01 smithi112 ceph-mon[32799]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:55:01.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:01 smithi019 ceph-mon[29209]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:55:01.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:01 smithi169 ceph-mon[33049]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T10:55:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.705+0000 7f212af0d640 1 -- 172.21.15.19:0/2945438532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 msgr2=0x7f2124101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.705+0000 7f212af0d640 1 --2- 172.21.15.19:0/2945438532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124101c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2118009900 tx=0x7f211802a410 comp rx=0 tx=0).stop 2024-04-25T10:55:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.705+0000 7f212af0d640 1 -- 172.21.15.19:0/2945438532 shutdown_connections 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.705+0000 7f212af0d640 1 --2- 172.21.15.19:0/2945438532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21241027e0 0x7f2124110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.705+0000 7f212af0d640 1 --2- 172.21.15.19:0/2945438532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2124109fe0 0x7f2124102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.705+0000 7f212af0d640 1 --2- 172.21.15.19:0/2945438532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124101c30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.705+0000 7f212af0d640 1 -- 172.21.15.19:0/2945438532 >> 172.21.15.19:0/2945438532 conn(0x7f21240fd880 msgr2=0x7f21240ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 -- 172.21.15.19:0/2945438532 shutdown_connections 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 -- 172.21.15.19:0/2945438532 wait complete. 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 Processor -- start 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 -- start start 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21241027e0 0x7f2124102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2124109fe0 0x7f2124103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2124113d30 con 0x7f2124109b40 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2124113bb0 con 0x7f2124109fe0 2024-04-25T10:55:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.706+0000 7f212af0d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2124113eb0 con 0x7f21241027e0 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124103350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57708/0 (socket says 172.21.15.19:57708) 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 -- 172.21.15.19:0/1278742146 learned_addr learned my addr 172.21.15.19:0/1278742146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f21294aa640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2124109fe0 0x7f2124103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 -- 172.21.15.19:0/1278742146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21241027e0 msgr2=0x7f2124102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2128ca9640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21241027e0 0x7f2124102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21241027e0 0x7f2124102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 -- 172.21.15.19:0/1278742146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2124109fe0 msgr2=0x7f2124103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2124109fe0 0x7f2124103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 -- 172.21.15.19:0/1278742146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21241a8ad0 con 0x7f2124109b40 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2123fff640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124103350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f211000d740 tx=0x7f211000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2121ffb640 1 -- 172.21.15.19:0/1278742146 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110017070 con 0x7f2124109b40 2024-04-25T10:55:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2121ffb640 1 -- 172.21.15.19:0/1278742146 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211000de50 con 0x7f2124109b40 2024-04-25T10:55:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f2121ffb640 1 -- 172.21.15.19:0/1278742146 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110003c20 con 0x7f2124109b40 2024-04-25T10:55:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.707+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21241a8dc0 con 0x7f2124109b40 2024-04-25T10:55:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.708+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a9280 con 0x7f2124109b40 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.710+0000 7f2121ffb640 1 -- 172.21.15.19:0/1278742146 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f2110003dc0 con 0x7f2124109b40 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.710+0000 7f2131dc1740 1 Processor -- start 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.710+0000 7f2131dc1740 1 -- start start 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a81030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148a81570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2131dc1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1489e7460 con 0x55b1486e11b0 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2131dc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1489a1590 con 0x55b1486c04d0 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2131dc1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b1489e3420 con 0x55b1486b82f0 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2121ffb640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f21140afd70 0x7f21140b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a80af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a80af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57722/0 (socket says 172.21.15.19:57722) 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 -- 172.21.15.19:0/952615627 learned_addr learned my addr 172.21.15.19:0/952615627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2128ca9640 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a81030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2123fff640 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148a81570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f21140afd70 0x7f21140b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2121ffb640 1 -- 172.21.15.19:0/1278742146 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f21100dedb0 con 0x7f2124109b40 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 -- 172.21.15.19:0/952615627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 msgr2=0x55b148a81030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a81030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 -- 172.21.15.19:0/952615627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 msgr2=0x55b148a81570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148a81570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 -- 172.21.15.19:0/952615627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1489ae0d0 con 0x55b1486e11b0 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f2123fff640 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148a81570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.711+0000 7f21294aa640 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a80af0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f211c0149c0 tx=0x7f211c014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deda8f68c293a46f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.712+0000 7f210effd640 1 -- 172.21.15.19:0/952615627 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c01eb40 con 0x55b1486e11b0 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.712+0000 7f210effd640 1 -- 172.21.15.19:0/952615627 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211c023070 con 0x55b1486e11b0 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.712+0000 7f210effd640 1 -- 172.21.15.19:0/952615627 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0138b0 con 0x55b1486e11b0 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 -- 172.21.15.19:0/952615627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 msgr2=0x55b148a80af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a80af0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f211c0149c0 tx=0x7f211c014e80 comp rx=0 tx=0).stop 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f21294aa640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f21140afd70 0x7f21140b2280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f211c022000 tx=0x7f211c020000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 -- 172.21.15.19:0/952615627 shutdown_connections 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148a81570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a81030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 --2- 172.21.15.19:0/952615627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a80af0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 -- 172.21.15.19:0/952615627 >> 172.21.15.19:0/952615627 conn(0x55b1486d5300 msgr2=0x55b148a7d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 -- 172.21.15.19:0/952615627 shutdown_connections 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 -- 172.21.15.19:0/952615627 wait complete. 2024-04-25T10:55:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 -- start start 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.713+0000 7f2131dc1740 1 Processor -- start 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a8cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486c04d0 0x55b148a8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486e11b0 0x55b148a8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b148a8dc20 con 0x55b1486c04d0 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b14896eb60 con 0x55b1486e11b0 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b148a8dda0 con 0x55b1486b82f0 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 10 client.0 _ll_register_callbacks cb 0x55b1489932b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486c04d0 0x55b148a8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486c04d0 0x55b148a8d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57736/0 (socket says 172.21.15.19:57736) 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 -- 172.21.15.19:0/550073547 learned_addr learned my addr 172.21.15.19:0/550073547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2128ca9640 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a8cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f21294aa640 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486e11b0 0x55b148a8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 -- 172.21.15.19:0/550073547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 msgr2=0x55b148a8cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a8cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 -- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486e11b0 msgr2=0x55b148a8afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486e11b0 0x55b148a8afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 -- 172.21.15.19:0/550073547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1486f9210 con 0x55b1486c04d0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2123fff640 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486c04d0 0x55b148a8d050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2110009610 tx=0x7f2110020c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f88963ee25f0364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2103fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2103fff640 1 -- 172.21.15.19:0/550073547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110020040 con 0x55b1486c04d0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2103fff640 1 -- 172.21.15.19:0/550073547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2110020cc0 con 0x55b1486c04d0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.714+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f211c023070 con 0x55b1486c04d0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2103fff640 1 -- 172.21.15.19:0/550073547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f21100a3300 con 0x55b1486c04d0 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 10 client.15471 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b1487c5620 con 0x55b1486c04d0 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2103fff640 1 -- 172.21.15.19:0/550073547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21100be070 con 0x55b1486c04d0 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2103fff640 1 -- 172.21.15.19:0/550073547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f21100a5f30 con 0x55b1486c04d0 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 10 client.15471 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 20 client.15471 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b14869fd80 0x55b148a9d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 4 client.15471 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1489f5070 con 0x55b14869fd80 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.715+0000 7f2128ca9640 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b14869fd80 0x55b148a9d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.716+0000 7f2128ca9640 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b14869fd80 0x55b148a9d790 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4291a051c34573 server_cookie=c6da1e5d9899fbfd in_seq=0 out_seq=0 2024-04-25T10:55:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.716+0000 7f2103fff640 10 client.15471 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2103fff640 1 -- 172.21.15.19:0/550073547 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1489f5070 con 0x55b14869fd80 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2103fff640 10 client.15471 handle_command_reply: tid=0 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 client.15471 shutdown 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486c04d0 msgr2=0x55b148a8d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486c04d0 0x55b148a8d050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2110009610 tx=0x7f2110020c50 comp rx=0 tx=0).stop 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 shutdown_connections 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b14869fd80 0x55b148a9d790 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486e11b0 0x55b148a8afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486c04d0 0x55b148a8d050 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2103fff640 0 client.15471 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 --2- 172.21.15.19:0/550073547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486b82f0 0x55b148a8cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 >> 172.21.15.19:0/550073547 conn(0x55b1486d5300 msgr2=0x55b148a80920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 shutdown_connections 2024-04-25T10:55:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 1 -- 172.21.15.19:0/550073547 wait complete. 2024-04-25T10:55:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.717+0000 7f2131dc1740 20 client.15471 trim_cache size 0 max 16384 2024-04-25T10:55:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 Processor -- start 2024-04-25T10:55:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 -- start start 2024-04-25T10:55:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486b82f0 0x55b148a84e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148aa2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486e11b0 0x55b148aa3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21241afb80 con 0x55b1486b82f0 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21241b2400 con 0x55b1486c04d0 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2131dc1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21241b01f0 con 0x55b1486e11b0 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486b82f0 0x55b148a84e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486b82f0 0x55b148a84e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57752/0 (socket says 172.21.15.19:57752) 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 -- 172.21.15.19:0/2434012184 learned_addr learned my addr 172.21.15.19:0/2434012184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 -- 172.21.15.19:0/2434012184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486e11b0 msgr2=0x55b148aa3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486e11b0 0x55b148aa3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 -- 172.21.15.19:0/2434012184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 msgr2=0x55b148aa2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2123fff640 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148aa2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148aa2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 -- 172.21.15.19:0/2434012184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b148a818a0 con 0x55b1486b82f0 2024-04-25T10:55:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2123fff640 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148aa2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.733+0000 7f2128ca9640 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486b82f0 0x55b148a84e50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2118009230 tx=0x7f211802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a62818ec8c65f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f210e7fc640 1 -- 172.21.15.19:0/2434012184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118035040 con 0x55b1486b82f0 2024-04-25T10:55:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f210e7fc640 1 -- 172.21.15.19:0/2434012184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2118047070 con 0x55b1486b82f0 2024-04-25T10:55:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f210e7fc640 1 -- 172.21.15.19:0/2434012184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118004920 con 0x55b1486b82f0 2024-04-25T10:55:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 -- 172.21.15.19:0/2434012184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486b82f0 msgr2=0x55b148a84e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486b82f0 0x55b148a84e50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2118009230 tx=0x7f211802ab90 comp rx=0 tx=0).stop 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 -- 172.21.15.19:0/2434012184 shutdown_connections 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486e11b0 0x55b148aa3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486c04d0 0x55b148aa2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2434012184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486b82f0 0x55b148a84e50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 -- 172.21.15.19:0/2434012184 >> 172.21.15.19:0/2434012184 conn(0x55b1486d5300 msgr2=0x55b148a84740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 -- 172.21.15.19:0/2434012184 shutdown_connections 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.734+0000 7f2131dc1740 1 -- 172.21.15.19:0/2434012184 wait complete. 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 -- start start 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 Processor -- start 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486b82f0 0x55b148a93180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486c04d0 0x55b148a936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a90600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21241b08c0 con 0x55b1486e11b0 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21241b1d30 con 0x55b1486b82f0 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21241b0f90 con 0x55b1486c04d0 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f2131dc1740 10 client.0 _ll_register_callbacks cb 0x7f2110005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f21294aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a90600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f21294aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a90600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57760/0 (socket says 172.21.15.19:57760) 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.735+0000 7f21294aa640 1 -- 172.21.15.19:0/2590505147 learned_addr learned my addr 172.21.15.19:0/2590505147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2123fff640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486c04d0 0x55b148a936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2128ca9640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486b82f0 0x55b148a93180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f21294aa640 1 -- 172.21.15.19:0/2590505147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486c04d0 msgr2=0x55b148a936c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f21294aa640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486c04d0 0x55b148a936c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f21294aa640 1 -- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486b82f0 msgr2=0x55b148a93180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f21294aa640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486b82f0 0x55b148a93180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f21294aa640 1 -- 172.21.15.19:0/2590505147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b148a90d40 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2128ca9640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486b82f0 0x55b148a93180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f21294aa640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a90600 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f211c012760 tx=0x7f211c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e494c9e63656cb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2103fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2103fff640 1 -- 172.21.15.19:0/2590505147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c013e00 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2103fff640 1 -- 172.21.15.19:0/2590505147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f211c014030 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2118047070 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2103fff640 1 -- 172.21.15.19:0/2590505147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f211c0094a0 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2131dc1740 10 client.15477 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.736+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b148b05500 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2103fff640 1 -- 172.21.15.19:0/2590505147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2118047070 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2103fff640 1 -- 172.21.15.19:0/2590505147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f211c00f040 con 0x55b1486e11b0 2024-04-25T10:55:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2131dc1740 10 client.15477 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2131dc1740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2131dc1740 20 client.15477 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b1486751c0 0x55b148b07b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2131dc1740 4 client.15477 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1489eeee0 con 0x55b1486751c0 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.737+0000 7f2128ca9640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b1486751c0 0x55b148b07b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.738+0000 7f2103fff640 10 client.15477 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.738+0000 7f2128ca9640 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b1486751c0 0x55b148b07b10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69a3be3353d82c2 server_cookie=19f85ea61a3aebec in_seq=0 out_seq=0 2024-04-25T10:55:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2103fff640 1 -- 172.21.15.19:0/2590505147 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1489eeee0 con 0x55b1486751c0 2024-04-25T10:55:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2103fff640 10 client.15477 handle_command_reply: tid=0 2024-04-25T10:55:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 client.15477 shutdown 2024-04-25T10:55:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 msgr2=0x55b148a90600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a90600 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f211c012760 tx=0x7f211c005f10 comp rx=0 tx=0).stop 2024-04-25T10:55:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 shutdown_connections 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b1486751c0 0x55b148b07b10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1486e11b0 0x55b148a90600 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1486c04d0 0x55b148a936c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2103fff640 0 client.15477 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 --2- 172.21.15.19:0/2590505147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1486b82f0 0x55b148a93180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 >> 172.21.15.19:0/2590505147 conn(0x55b1486d5300 msgr2=0x55b148a80880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 shutdown_connections 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 1 -- 172.21.15.19:0/2590505147 wait complete. 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.746+0000 7f2131dc1740 20 client.15477 trim_cache size 0 max 16384 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:55:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:55:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:55:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f21140afd70 msgr2=0x7f21140b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f21140afd70 0x7f21140b2280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f211c022000 tx=0x7f211c020000 comp rx=0 tx=0).stop 2024-04-25T10:55:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 msgr2=0x7f2124103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124103350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f211000d740 tx=0x7f211000dc00 comp rx=0 tx=0).stop 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 shutdown_connections 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f21140afd70 0x7f21140b2280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2124109fe0 0x7f2124103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2124109b40 0x7f2124103350 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 --2- 172.21.15.19:0/1278742146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21241027e0 0x7f2124102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 >> 172.21.15.19:0/1278742146 conn(0x7f21240fd880 msgr2=0x7f21240fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 shutdown_connections 2024-04-25T10:55:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:02.747+0000 7f212af0d640 1 -- 172.21.15.19:0/1278742146 wait complete. 2024-04-25T10:55:03.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:03 smithi112 ceph-mon[32799]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T10:55:03.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:03 smithi112 ceph-mon[32799]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-04-25T10:55:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:03 smithi019 ceph-mon[29209]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T10:55:03.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:03 smithi019 ceph-mon[29209]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-04-25T10:55:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:03 smithi169 ceph-mon[33049]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T10:55:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:03 smithi169 ceph-mon[33049]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-04-25T10:55:04.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:55:04.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:55:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 -- 172.21.15.19:0/2020466797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 msgr2=0x7fa5181025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 --2- 172.21.15.19:0/2020466797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa5181025b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa508009900 tx=0x7fa50802a410 comp rx=0 tx=0).stop 2024-04-25T10:55:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 -- 172.21.15.19:0/2020466797 shutdown_connections 2024-04-25T10:55:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 --2- 172.21.15.19:0/2020466797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51810d380 0x7fa51810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 --2- 172.21.15.19:0/2020466797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa518102af0 0x7fa518102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 --2- 172.21.15.19:0/2020466797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa5181025b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 -- 172.21.15.19:0/2020466797 >> 172.21.15.19:0/2020466797 conn(0x7fa518078100 msgr2=0x7fa518078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 -- 172.21.15.19:0/2020466797 shutdown_connections 2024-04-25T10:55:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.443+0000 7fa51edd7640 1 -- 172.21.15.19:0/2020466797 wait complete. 2024-04-25T10:55:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 Processor -- start 2024-04-25T10:55:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 -- start start 2024-04-25T10:55:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa518102af0 0x7fa51819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa51819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51810d380 0x7fa51819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa518112be0 con 0x7fa518108a20 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa518112a60 con 0x7fa51810d380 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51edd7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa518112d60 con 0x7fa518102af0 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa51819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa51819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57784/0 (socket says 172.21.15.19:57784) 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa517fff640 1 -- 172.21.15.19:0/3956340358 learned_addr learned my addr 172.21.15.19:0/3956340358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51cb73640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa518102af0 0x7fa51819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.444+0000 7fa51d374640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51810d380 0x7fa51819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa517fff640 1 -- 172.21.15.19:0/3956340358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa518102af0 msgr2=0x7fa51819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa517fff640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa518102af0 0x7fa51819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa517fff640 1 -- 172.21.15.19:0/3956340358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51810d380 msgr2=0x7fa51819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa517fff640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51810d380 0x7fa51819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa517fff640 1 -- 172.21.15.19:0/3956340358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51819f570 con 0x7fa518108a20 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa51d374640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51810d380 0x7fa51819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa517fff640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa51819ac70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005d00 tx=0x7fa50c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa515ffb640 1 -- 172.21.15.19:0/3956340358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c017070 con 0x7fa518108a20 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa515ffb640 1 -- 172.21.15.19:0/3956340358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa50c009c30 con 0x7fa518108a20 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51819f860 con 0x7fa518108a20 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa515ffb640 1 -- 172.21.15.19:0/3956340358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00f620 con 0x7fa518108a20 2024-04-25T10:55:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.445+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51819fd60 con 0x7fa518108a20 2024-04-25T10:55:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.448+0000 7fa525d63740 1 Processor -- start 2024-04-25T10:55:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.448+0000 7fa525d63740 1 -- start start 2024-04-25T10:55:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa525d63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd673ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa525d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd63724d0 0x557fd673bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa525d63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd67488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa525d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557fd66a1d80 con 0x557fd63724d0 2024-04-25T10:55:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa525d63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557fd665b8b0 con 0x557fd639b390 2024-04-25T10:55:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa525d63740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557fd669dd40 con 0x557fd637a6b0 2024-04-25T10:55:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa515ffb640 1 -- 172.21.15.19:0/3956340358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7fa50c01a070 con 0x7fa518108a20 2024-04-25T10:55:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd63724d0 0x557fd673bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd63724d0 0x557fd673bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57796/0 (socket says 172.21.15.19:57796) 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 -- 172.21.15.19:0/240741738 learned_addr learned my addr 172.21.15.19:0/240741738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51d374640 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd673ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 -- 172.21.15.19:0/240741738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 msgr2=0x557fd67488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa517fff640 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd67488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd67488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 -- 172.21.15.19:0/240741738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 msgr2=0x557fd673ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd673ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa51cb73640 1 -- 172.21.15.19:0/240741738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd662ca40 con 0x557fd63724d0 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.449+0000 7fa517fff640 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd67488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.450+0000 7fa51cb73640 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd63724d0 0x557fd673bfa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa508009900 tx=0x7fa508005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca0d6d2746f32bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.450+0000 7fa515ffb640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa5000afd80 0x7fa5000b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.450+0000 7fa4feffd640 1 -- 172.21.15.19:0/240741738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508040070 con 0x557fd63724d0 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.450+0000 7fa4feffd640 1 -- 172.21.15.19:0/240741738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508004030 con 0x557fd63724d0 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.450+0000 7fa4feffd640 1 -- 172.21.15.19:0/240741738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508035040 con 0x557fd63724d0 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.450+0000 7fa51d374640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa5000afd80 0x7fa5000b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.450+0000 7fa515ffb640 1 -- 172.21.15.19:0/3956340358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa50c0ded40 con 0x7fa518108a20 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 -- 172.21.15.19:0/240741738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd63724d0 msgr2=0x557fd673bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa51d374640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa5000afd80 0x7fa5000b2290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x557fd6738bd0 tx=0x7fa5100098a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd63724d0 0x557fd673bfa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa508009900 tx=0x7fa508005ef0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 -- 172.21.15.19:0/240741738 shutdown_connections 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd67488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd63724d0 0x557fd673bfa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 --2- 172.21.15.19:0/240741738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd673ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 -- 172.21.15.19:0/240741738 >> 172.21.15.19:0/240741738 conn(0x557fd638f4e0 msgr2=0x557fd6738210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 -- 172.21.15.19:0/240741738 shutdown_connections 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 -- 172.21.15.19:0/240741738 wait complete. 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 -- start start 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.451+0000 7fa525d63740 1 Processor -- start 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa525d63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6746660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa525d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd6746ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa525d63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd674c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa525d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557fd6748f80 con 0x557fd637a6b0 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa525d63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557fd6628d10 con 0x557fd639b390 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa525d63740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557fd6749100 con 0x557fd63724d0 2024-04-25T10:55:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa525d63740 10 client.0 _ll_register_callbacks cb 0x557fd6655210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd6746ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd6746ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57806/0 (socket says 172.21.15.19:57806) 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 -- 172.21.15.19:0/3386429541 learned_addr learned my addr 172.21.15.19:0/3386429541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa51d374640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd674c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa51cb73640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6746660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 -- 172.21.15.19:0/3386429541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 msgr2=0x557fd6746660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6746660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 -- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 msgr2=0x557fd674c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd674c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa517fff640 1 -- 172.21.15.19:0/3386429541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd647d7d0 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.452+0000 7fa51d374640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd674c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa517fff640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd6746ba0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0a6c50 tx=0x7fa50c0a7220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d206d726b6164c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa4f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3386429541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c021070 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3386429541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa50c0a6130 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa508004030 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3386429541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa50c0a67d0 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 10 client.15489 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557fd63b33f0 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3386429541 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508004030 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3386429541 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa50c0a5990 con 0x557fd637a6b0 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 10 client.15489 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 20 client.15489 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd6359f60 0x557fd67586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 4 client.15489 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.453+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fd66af990 con 0x557fd6359f60 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.454+0000 7fa51cb73640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd6359f60 0x557fd67586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.454+0000 7fa4f7fff640 10 client.15489 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.454+0000 7fa51cb73640 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd6359f60 0x557fd67586a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b6bb84fcc9fcab server_cookie=20fab569d3591a in_seq=0 out_seq=0 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3386429541 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557fd66af990 con 0x557fd6359f60 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa4f7fff640 10 client.15489 handle_command_reply: tid=0 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 client.15489 shutdown 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 msgr2=0x557fd6746ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd6746ba0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0a6c50 tx=0x7fa50c0a7220 comp rx=0 tx=0).stop 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 shutdown_connections 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd6359f60 0x557fd67586a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd639b390 0x557fd674c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa4f7fff640 0 client.15489 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd6746ba0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 --2- 172.21.15.19:0/3386429541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6746660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 >> 172.21.15.19:0/3386429541 conn(0x557fd638f4e0 msgr2=0x557fd673d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 shutdown_connections 2024-04-25T10:55:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 1 -- 172.21.15.19:0/3386429541 wait complete. 2024-04-25T10:55:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.455+0000 7fa525d63740 20 client.15489 trim_cache size 0 max 16384 2024-04-25T10:55:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.476+0000 7fa525d63740 1 Processor -- start 2024-04-25T10:55:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.476+0000 7fa525d63740 1 -- start start 2024-04-25T10:55:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa525d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd638f4e0 0x557fd6741c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa525d63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd63724d0 0x557fd6745430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa525d63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd675de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa525d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181ad770 con 0x557fd638f4e0 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa525d63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181afff0 con 0x557fd63724d0 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa525d63740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181adde0 con 0x557fd637a6b0 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd638f4e0 0x557fd6741c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd638f4e0 0x557fd6741c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57810/0 (socket says 172.21.15.19:57810) 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51d374640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd675de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 -- 172.21.15.19:0/2997161756 learned_addr learned my addr 172.21.15.19:0/2997161756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa517fff640 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd63724d0 0x557fd6745430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 -- 172.21.15.19:0/2997161756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 msgr2=0x557fd675de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd675de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 -- 172.21.15.19:0/2997161756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd63724d0 msgr2=0x557fd6745430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd63724d0 0x557fd6745430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.477+0000 7fa51cb73640 1 -- 172.21.15.19:0/2997161756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd63730f0 con 0x557fd638f4e0 2024-04-25T10:55:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.478+0000 7fa51cb73640 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd638f4e0 0x557fd6741c50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa50802a930 tx=0x7fa508005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dc2f1a33c3d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.478+0000 7fa4fe7fc640 1 -- 172.21.15.19:0/2997161756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508055070 con 0x557fd638f4e0 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.478+0000 7fa4fe7fc640 1 -- 172.21.15.19:0/2997161756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa50804fe50 con 0x557fd638f4e0 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.478+0000 7fa4fe7fc640 1 -- 172.21.15.19:0/2997161756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508040040 con 0x557fd638f4e0 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 -- 172.21.15.19:0/2997161756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd638f4e0 msgr2=0x557fd6741c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd638f4e0 0x557fd6741c50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa50802a930 tx=0x7fa508005fd0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 -- 172.21.15.19:0/2997161756 shutdown_connections 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd637a6b0 0x557fd675de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd63724d0 0x557fd6745430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 --2- 172.21.15.19:0/2997161756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd638f4e0 0x557fd6741c50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 -- 172.21.15.19:0/2997161756 >> 172.21.15.19:0/2997161756 conn(0x557fd673bfc0 msgr2=0x557fd6744d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 -- 172.21.15.19:0/2997161756 shutdown_connections 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 -- 172.21.15.19:0/2997161756 wait complete. 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 -- start start 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.479+0000 7fa525d63740 1 Processor -- start 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa525d63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6752470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa525d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd674d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa525d63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd638f4e0 0x557fd674dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa525d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181ae4b0 con 0x557fd637a6b0 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa525d63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181af920 con 0x557fd638f4e0 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa525d63740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181aeb80 con 0x557fd63724d0 2024-04-25T10:55:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa525d63740 10 client.0 _ll_register_callbacks cb 0x7fa50c010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd674d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd674d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57822/0 (socket says 172.21.15.19:57822) 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 -- 172.21.15.19:0/3947259782 learned_addr learned my addr 172.21.15.19:0/3947259782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa51d374640 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd638f4e0 0x557fd674dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa51cb73640 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6752470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 -- 172.21.15.19:0/3947259782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 msgr2=0x557fd6752470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6752470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 -- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd638f4e0 msgr2=0x557fd674dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd638f4e0 0x557fd674dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.480+0000 7fa517fff640 1 -- 172.21.15.19:0/3947259782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd66624b0 con 0x557fd637a6b0 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa517fff640 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd674d7c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0a5db0 tx=0x7fa50c0a6b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59a47bcd150e90b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa4f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3947259782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c014020 con 0x557fd637a6b0 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3947259782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa50c0a2840 con 0x557fd637a6b0 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa50804fe50 con 0x557fd637a6b0 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3947259782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa50c021070 con 0x557fd637a6b0 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 10 client.15495 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557fd67c04e0 con 0x557fd637a6b0 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3947259782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa50804fe50 con 0x557fd637a6b0 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3947259782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa50c0a2040 con 0x557fd637a6b0 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 10 client.15495 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 20 client.15495 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd639b390 0x557fd67c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 4 client.15495 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.481+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557fd66a9800 con 0x557fd639b390 2024-04-25T10:55:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa51cb73640 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd639b390 0x557fd67c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa51cb73640 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd639b390 0x557fd67c2c10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6683fb3c2bb804be server_cookie=c3de1343421033d in_seq=0 out_seq=0 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa4f7fff640 10 client.15495 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa4f7fff640 1 -- 172.21.15.19:0/3947259782 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557fd66a9800 con 0x557fd639b390 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa4f7fff640 10 client.15495 handle_command_reply: tid=0 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa525d63740 1 client.15495 shutdown 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 msgr2=0x557fd674d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa525d63740 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd674d7c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0a5db0 tx=0x7fa50c0a6b00 comp rx=0 tx=0).stop 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 shutdown_connections 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa525d63740 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557fd639b390 0x557fd67c2c10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa525d63740 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd638f4e0 0x557fd674dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.482+0000 7fa525d63740 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd637a6b0 0x557fd674d7c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa4f7fff640 0 client.15495 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa525d63740 1 --2- 172.21.15.19:0/3947259782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fd63724d0 0x557fd6752470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 >> 172.21.15.19:0/3947259782 conn(0x557fd673bfc0 msgr2=0x557fd6747a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 shutdown_connections 2024-04-25T10:55:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa525d63740 1 -- 172.21.15.19:0/3947259782 wait complete. 2024-04-25T10:55:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa525d63740 20 client.15495 trim_cache size 0 max 16384 2024-04-25T10:55:04.489 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:55:04.489 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:55:04.489 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:55:04.489 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:55:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa5000afd80 msgr2=0x7fa5000b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa5000afd80 0x7fa5000b2290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x557fd6738bd0 tx=0x7fa5100098a0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 msgr2=0x7fa51819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa51819ac70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005d00 tx=0x7fa50c00dde0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 shutdown_connections 2024-04-25T10:55:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa5000afd80 0x7fa5000b2290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51810d380 0x7fa51819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518108a20 0x7fa51819ac70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 --2- 172.21.15.19:0/3956340358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa518102af0 0x7fa51819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.483+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 >> 172.21.15.19:0/3956340358 conn(0x7fa518078100 msgr2=0x7fa518069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.484+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 shutdown_connections 2024-04-25T10:55:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.484+0000 7fa51edd7640 1 -- 172.21.15.19:0/3956340358 wait complete. 2024-04-25T10:55:04.495 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:55:04.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 -- 172.21.15.19:0/2661536514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00107260 msgr2=0x7f3c00107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 --2- 172.21.15.19:0/2661536514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00107260 0x7f3c00107670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009900 tx=0x7f3bf402a410 comp rx=0 tx=0).stop 2024-04-25T10:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 -- 172.21.15.19:0/2661536514 shutdown_connections 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 --2- 172.21.15.19:0/2661536514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c00107ce0 0x7f3c0010f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 --2- 172.21.15.19:0/2661536514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c00107260 0x7f3c00107670 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 --2- 172.21.15.19:0/2661536514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c001033a0 0x7f3c001037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 -- 172.21.15.19:0/2661536514 >> 172.21.15.19:0/2661536514 conn(0x7f3c00077890 msgr2=0x7f3c00077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 -- 172.21.15.19:0/2661536514 shutdown_connections 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.607+0000 7f3c06a61640 1 -- 172.21.15.19:0/2661536514 wait complete. 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.608+0000 7f3c06a61640 1 Processor -- start 2024-04-25T10:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.608+0000 7f3c06a61640 1 -- start start 2024-04-25T10:55:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.608+0000 7f3c06a61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c001033a0 0x7f3c00199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.608+0000 7f3c06a61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c00107260 0x7f3c00199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c06a61640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c00107ce0 0x7f3c0019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c06a61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00112be0 con 0x7f3c001033a0 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c06a61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00112a60 con 0x7f3c00107260 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c06a61640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00112d60 con 0x7f3c00107ce0 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c001033a0 0x7f3c00199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c001033a0 0x7f3c00199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57834/0 (socket says 172.21.15.19:57834) 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 -- 172.21.15.19:0/4044048916 learned_addr learned my addr 172.21.15.19:0/4044048916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bff7fe640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c00107260 0x7f3c00199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c04ffe640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c00107ce0 0x7f3c0019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 -- 172.21.15.19:0/4044048916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c00107ce0 msgr2=0x7f3c0019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c00107ce0 0x7f3c0019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 -- 172.21.15.19:0/4044048916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c00107260 msgr2=0x7f3c00199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c00107260 0x7f3c00199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 -- 172.21.15.19:0/4044048916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0019e460 con 0x7f3c001033a0 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c04ffe640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c00107ce0 0x7f3c0019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bff7fe640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c00107260 0x7f3c00199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfffff640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c001033a0 0x7f3c00199600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d770 tx=0x7f3bf000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfd7fa640 1 -- 172.21.15.19:0/4044048916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0017070 con 0x7f3c001033a0 2024-04-25T10:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0019e750 con 0x7f3c001033a0 2024-04-25T10:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfd7fa640 1 -- 172.21.15.19:0/4044048916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf0009c30 con 0x7f3c001033a0 2024-04-25T10:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3bfd7fa640 1 -- 172.21.15.19:0/4044048916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000f620 con 0x7f3c001033a0 2024-04-25T10:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.609+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c00101bd0 con 0x7f3c001033a0 2024-04-25T10:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.611+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c00103f00 con 0x7f3c001033a0 2024-04-25T10:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.612+0000 7f3bfd7fa640 1 -- 172.21.15.19:0/4044048916 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f3bf001a070 con 0x7f3c001033a0 2024-04-25T10:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.613+0000 7f3bfd7fa640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3be80afd80 0x7f3be80b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.613+0000 7f3bfd7fa640 1 -- 172.21.15.19:0/4044048916 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3bf00deed0 con 0x7f3c001033a0 2024-04-25T10:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.613+0000 7f3bff7fe640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3be80afd80 0x7f3be80b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.614+0000 7f3bff7fe640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3be80afd80 0x7f3be80b2290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009230 tx=0x7f3bf4000bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.617+0000 7f3bfd7fa640 1 -- 172.21.15.19:0/4044048916 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bf0009de0 con 0x7f3c001033a0 2024-04-25T10:55:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.845+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c001025f0 con 0x7f3c001033a0 2024-04-25T10:55:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.846+0000 7f3bfd7fa640 1 -- 172.21.15.19:0/4044048916 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f3bf00a74f0 con 0x7f3c001033a0 2024-04-25T10:55:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:55:04.849 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:55:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:55:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.848+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3be80afd80 msgr2=0x7f3be80b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.848+0000 7f3c06a61640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3be80afd80 0x7f3be80b2290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009230 tx=0x7f3bf4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.848+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c001033a0 msgr2=0x7f3c00199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.848+0000 7f3c06a61640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c001033a0 0x7f3c00199600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d770 tx=0x7f3bf000dc30 comp rx=0 tx=0).stop 2024-04-25T10:55:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 shutdown_connections 2024-04-25T10:55:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3be80afd80 0x7f3be80b2290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c00107ce0 0x7f3c0019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c00107260 0x7f3c00199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 --2- 172.21.15.19:0/4044048916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c001033a0 0x7f3c00199600 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 >> 172.21.15.19:0/4044048916 conn(0x7f3c00077890 msgr2=0x7f3c000ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 shutdown_connections 2024-04-25T10:55:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.849+0000 7f3c06a61640 1 -- 172.21.15.19:0/4044048916 wait complete. 2024-04-25T10:55:04.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 -- 172.21.15.19:0/905593353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac109b10 msgr2=0x7f43ac1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 --2- 172.21.15.19:0/905593353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac109b10 0x7f43ac1036b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009900 tx=0x7f43a802a410 comp rx=0 tx=0).stop 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 -- 172.21.15.19:0/905593353 shutdown_connections 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 --2- 172.21.15.19:0/905593353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac104260 0x7f43ac110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 --2- 172.21.15.19:0/905593353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac109fb0 0x7f43ac103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 --2- 172.21.15.19:0/905593353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac109b10 0x7f43ac1036b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 -- 172.21.15.19:0/905593353 >> 172.21.15.19:0/905593353 conn(0x7f43ac0ff210 msgr2=0x7f43ac101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 -- 172.21.15.19:0/905593353 shutdown_connections 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.975+0000 7f43b36e7640 1 -- 172.21.15.19:0/905593353 wait complete. 2024-04-25T10:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 Processor -- start 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 -- start start 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac104260 0x7f43ac19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 0x7f43ac19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac109fb0 0x7f43ac19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac113d00 con 0x7f43ac104260 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac113b80 con 0x7f43ac109b10 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b36e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac113e80 con 0x7f43ac109fb0 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac104260 0x7f43ac19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac104260 0x7f43ac19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57856/0 (socket says 172.21.15.19:57856) 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 -- 172.21.15.19:0/1487714291 learned_addr learned my addr 172.21.15.19:0/1487714291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac109fb0 0x7f43ac19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b0c82640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 0x7f43ac19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 -- 172.21.15.19:0/1487714291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac109fb0 msgr2=0x7f43ac19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac109fb0 0x7f43ac19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 -- 172.21.15.19:0/1487714291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 msgr2=0x7f43ac19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 0x7f43ac19ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1483640 1 -- 172.21.15.19:0/1487714291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac19f5e0 con 0x7f43ac104260 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac109fb0 0x7f43ac19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.976+0000 7f43b0c82640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 0x7f43ac19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.977+0000 7f43b1483640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac104260 0x7f43ac19a780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f43a8032fd0 tx=0x7f43a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.977+0000 7f43a27fc640 1 -- 172.21.15.19:0/1487714291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a803f030 con 0x7f43ac104260 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.977+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac19f870 con 0x7f43ac104260 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.977+0000 7f43a27fc640 1 -- 172.21.15.19:0/1487714291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8004490 con 0x7f43ac104260 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.977+0000 7f43a27fc640 1 -- 172.21.15.19:0/1487714291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8035070 con 0x7f43ac104260 2024-04-25T10:55:04.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.977+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac19fdb0 con 0x7f43ac104260 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.979+0000 7f43ba597740 1 Processor -- start 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43ba597740 1 -- start start 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43a27fc640 1 -- 172.21.15.19:0/1487714291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f43a8042030 con 0x7f43ac104260 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43ba597740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822ce7390 0x55a823088810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43ba597740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cbe4d0 0x55a823088d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43ba597740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a8230956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43ba597740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a822fedd80 con 0x55a822cc66b0 2024-04-25T10:55:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43ba597740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a822fa7a20 con 0x55a822ce7390 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43ba597740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a822fe9d40 con 0x55a822cbe4d0 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a8230956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a8230956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57872/0 (socket says 172.21.15.19:57872) 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 -- 172.21.15.19:0/3478162374 learned_addr learned my addr 172.21.15.19:0/3478162374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b1c84640 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822ce7390 0x55a823088810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b1483640 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cbe4d0 0x55a823088d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 -- 172.21.15.19:0/3478162374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cbe4d0 msgr2=0x55a823088d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cbe4d0 0x55a823088d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 -- 172.21.15.19:0/3478162374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822ce7390 msgr2=0x55a823088810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822ce7390 0x55a823088810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b0c82640 1 -- 172.21.15.19:0/3478162374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a822fb2c30 con 0x55a822cc66b0 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43b1c84640 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822ce7390 0x55a823088810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.980+0000 7f43a27fc640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f43980afd80 0x7f43980b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.981+0000 7f43b0c82640 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a8230956a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f439c00bc60 tx=0x7f439c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d002e6ce9cf5c90c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.981+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f43980afd80 0x7f43980b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.981+0000 7f43a27fc640 1 -- 172.21.15.19:0/1487714291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f43a80ff7a0 con 0x7f43ac104260 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.981+0000 7f43977fe640 1 -- 172.21.15.19:0/3478162374 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c015070 con 0x55a822cc66b0 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.981+0000 7f43977fe640 1 -- 172.21.15.19:0/3478162374 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439c01a910 con 0x55a822cc66b0 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.981+0000 7f43977fe640 1 -- 172.21.15.19:0/3478162374 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c011900 con 0x55a822cc66b0 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 -- 172.21.15.19:0/3478162374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 msgr2=0x55a8230956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f43980afd80 0x7f43980b2290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43a40095f0 tx=0x7f43a4009580 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a8230956a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f439c00bc60 tx=0x7f439c00f730 comp rx=0 tx=0).stop 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 -- 172.21.15.19:0/3478162374 shutdown_connections 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a8230956a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cbe4d0 0x55a823088d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 --2- 172.21.15.19:0/3478162374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822ce7390 0x55a823088810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 -- 172.21.15.19:0/3478162374 >> 172.21.15.19:0/3478162374 conn(0x55a822cdb4e0 msgr2=0x55a823084fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 -- 172.21.15.19:0/3478162374 shutdown_connections 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 -- 172.21.15.19:0/3478162374 wait complete. 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 -- start start 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.982+0000 7f43ba597740 1 Processor -- start 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43ba597740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a823093270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43ba597740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a8230937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43ba597740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ce7390 0x55a82309f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43ba597740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a823095d90 con 0x55a822ce7390 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43ba597740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a822f74dc0 con 0x55a822cbe4d0 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43ba597740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a823095f10 con 0x55a822cc66b0 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43ba597740 10 client.0 _ll_register_callbacks cb 0x55a822f90600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ce7390 0x55a82309f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ce7390 0x55a82309f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57878/0 (socket says 172.21.15.19:57878) 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 -- 172.21.15.19:0/2498009901 learned_addr learned my addr 172.21.15.19:0/2498009901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1483640 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a823093270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b0c82640 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a8230937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 -- 172.21.15.19:0/2498009901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 msgr2=0x55a8230937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a8230937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 -- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 msgr2=0x55a823093270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a823093270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 -- 172.21.15.19:0/2498009901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a822cff3f0 con 0x55a822ce7390 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f43b1c84640 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ce7390 0x55a82309f210 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009fd0 tx=0x7f43a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfb85b660216aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.983+0000 7f4394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f4394ff9640 1 -- 172.21.15.19:0/2498009901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a402c060 con 0x55a822ce7390 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f4394ff9640 1 -- 172.21.15.19:0/2498009901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f43a40107f0 con 0x55a822ce7390 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439c01a910 con 0x55a822ce7390 2024-04-25T10:55:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f4394ff9640 1 -- 172.21.15.19:0/2498009901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f43a40109f0 con 0x55a822ce7390 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 10 client.15513 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a822dc97d0 con 0x55a822ce7390 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f4394ff9640 1 -- 172.21.15.19:0/2498009901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439c01a910 con 0x55a822ce7390 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f4394ff9640 1 -- 172.21.15.19:0/2498009901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f43a400f040 con 0x55a822ce7390 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 10 client.15513 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 20 client.15513 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ca5f60 0x55a8230a5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 4 client.15513 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.984+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a822ffb990 con 0x55a822ca5f60 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.985+0000 7f43b1483640 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ca5f60 0x55a8230a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.985+0000 7f4394ff9640 10 client.15513 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.985+0000 7f43b1483640 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ca5f60 0x55a8230a5750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d3294d112e0a00 server_cookie=698aa56474b7f10e in_seq=0 out_seq=0 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f4394ff9640 1 -- 172.21.15.19:0/2498009901 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a822ffb990 con 0x55a822ca5f60 2024-04-25T10:55:04.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f4394ff9640 10 client.15513 handle_command_reply: tid=0 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 client.15513 shutdown 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ce7390 msgr2=0x55a82309f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ce7390 0x55a82309f210 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009fd0 tx=0x7f43a4004a40 comp rx=0 tx=0).stop 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 shutdown_connections 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ca5f60 0x55a8230a5750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ce7390 0x55a82309f210 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a8230937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f4394ff9640 0 client.15513 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 --2- 172.21.15.19:0/2498009901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a823093270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 >> 172.21.15.19:0/2498009901 conn(0x55a822cdb4e0 msgr2=0x55a82308a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 shutdown_connections 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 1 -- 172.21.15.19:0/2498009901 wait complete. 2024-04-25T10:55:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:04.986+0000 7f43ba597740 20 client.15513 trim_cache size 0 max 16384 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 Processor -- start 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 -- start start 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ca5f60 0x55a82309ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a823088c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1ad8d0 con 0x55a822ca5f60 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1b0150 con 0x55a822cbe4d0 2024-04-25T10:55:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43ba597740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1adf40 con 0x55a822cc66b0 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43b1483640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ca5f60 0x55a82309ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43b1483640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ca5f60 0x55a82309ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57886/0 (socket says 172.21.15.19:57886) 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43b1483640 1 -- 172.21.15.19:0/1770969834 learned_addr learned my addr 172.21.15.19:0/1770969834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a823088c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.000+0000 7f43b1483640 1 -- 172.21.15.19:0/1770969834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 msgr2=0x55a823088c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f43b1483640 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a823088c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f43b1483640 1 -- 172.21.15.19:0/1770969834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 msgr2=0x55a8230886f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f43b1483640 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f43b1483640 1 -- 172.21.15.19:0/1770969834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a823084620 con 0x55a822ca5f60 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a823088c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f43b1483640 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ca5f60 0x55a82309ee70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f43a80c3870 tx=0x7f43a80c3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebeb97a1c0ed24b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f4396ffd640 1 -- 172.21.15.19:0/1770969834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8049040 con 0x55a822ca5f60 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f4396ffd640 1 -- 172.21.15.19:0/1770969834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a80c7ef0 con 0x55a822ca5f60 2024-04-25T10:55:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.001+0000 7f4396ffd640 1 -- 172.21.15.19:0/1770969834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a803b460 con 0x55a822ca5f60 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 -- 172.21.15.19:0/1770969834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ca5f60 msgr2=0x55a82309ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ca5f60 0x55a82309ee70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f43a80c3870 tx=0x7f43a80c3040 comp rx=0 tx=0).stop 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 -- 172.21.15.19:0/1770969834 shutdown_connections 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822cc66b0 0x55a823088c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230886f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 --2- 172.21.15.19:0/1770969834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822ca5f60 0x55a82309ee70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 -- 172.21.15.19:0/1770969834 >> 172.21.15.19:0/1770969834 conn(0x55a822cdb4e0 msgr2=0x55a82308e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 -- 172.21.15.19:0/1770969834 shutdown_connections 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 -- 172.21.15.19:0/1770969834 wait complete. 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 -- start start 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.002+0000 7f43ba597740 1 Processor -- start 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822ca5f60 0x55a8230961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a823096c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1ae610 con 0x55a822cc66b0 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1afa80 con 0x55a822cbe4d0 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1aece0 con 0x55a822ca5f60 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 10 client.0 _ll_register_callbacks cb 0x7f43a401d590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a823096c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a823096c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57896/0 (socket says 172.21.15.19:57896) 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 -- 172.21.15.19:0/1515255813 learned_addr learned my addr 172.21.15.19:0/1515255813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b0c82640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1483640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822ca5f60 0x55a8230961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 -- 172.21.15.19:0/1515255813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822ca5f60 msgr2=0x55a8230961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822ca5f60 0x55a8230961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 -- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 msgr2=0x55a8230966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 -- 172.21.15.19:0/1515255813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82310bcb0 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b0c82640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43b1c84640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a823096c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f43a4000f80 tx=0x7f43a4010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938ed414bdd22e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f4394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f4394ff9640 1 -- 172.21.15.19:0/1515255813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4010d90 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f4394ff9640 1 -- 172.21.15.19:0/1515255813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f43a4010f30 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.003+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a80c7ef0 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f4394ff9640 1 -- 172.21.15.19:0/1515255813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f43a4011130 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 10 client.15519 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a82310ce80 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f4394ff9640 1 -- 172.21.15.19:0/1515255813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a80c7ef0 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f4394ff9640 1 -- 172.21.15.19:0/1515255813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f43a4031720 con 0x55a822cc66b0 2024-04-25T10:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 10 client.15519 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 20 client.15519 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ce7390 0x55a82310f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 4 client.15519 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a822ff5800 con 0x55a822ce7390 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.004+0000 7f43b1483640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ce7390 0x55a82310f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f4394ff9640 10 client.15519 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43b1483640 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ce7390 0x55a82310f600 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d709419be7e5a8 server_cookie=a07d719e956c775 in_seq=0 out_seq=0 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f4394ff9640 1 -- 172.21.15.19:0/1515255813 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a822ff5800 con 0x55a822ce7390 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f4394ff9640 10 client.15519 handle_command_reply: tid=0 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 client.15519 shutdown 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 msgr2=0x55a823096c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a823096c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f43a4000f80 tx=0x7f43a4010430 comp rx=0 tx=0).stop 2024-04-25T10:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 shutdown_connections 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a822ce7390 0x55a82310f600 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a822cc66b0 0x55a823096c30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a822cbe4d0 0x55a8230966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 --2- 172.21.15.19:0/1515255813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a822ca5f60 0x55a8230961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 >> 172.21.15.19:0/1515255813 conn(0x55a822cdb4e0 msgr2=0x55a82308e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f4394ff9640 0 client.15519 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 shutdown_connections 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 1 -- 172.21.15.19:0/1515255813 wait complete. 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.005+0000 7f43ba597740 20 client.15519 trim_cache size 0 max 16384 2024-04-25T10:55:05.012 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:55:05.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f43980afd80 msgr2=0x7f43980b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:05.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f43980afd80 0x7f43980b2290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43a40095f0 tx=0x7f43a4009580 comp rx=0 tx=0).stop 2024-04-25T10:55:05.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac104260 msgr2=0x7f43ac19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:05.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac104260 0x7f43ac19a780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f43a8032fd0 tx=0x7f43a8004270 comp rx=0 tx=0).stop 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 shutdown_connections 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f43980afd80 0x7f43980b2290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac109fb0 0x7f43ac19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac109b10 0x7f43ac19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 --2- 172.21.15.19:0/1487714291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43ac104260 0x7f43ac19a780 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 >> 172.21.15.19:0/1487714291 conn(0x7f43ac0ff210 msgr2=0x7f43ac0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 shutdown_connections 2024-04-25T10:55:05.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:05.006+0000 7f43b36e7640 1 -- 172.21.15.19:0/1487714291 wait complete. 2024-04-25T10:55:05.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:05 smithi019 ceph-mon[29209]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:55:05.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:05 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4044048916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:55:05.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:05 smithi112 ceph-mon[32799]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:55:05.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:05 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4044048916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:55:05.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:05 smithi169 ceph-mon[33049]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:55:05.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:05 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4044048916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:55:06.018 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:55:06.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:55:06.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:55:06.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.149+0000 7f8060384640 1 -- 172.21.15.19:0/3556483185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80581032c0 msgr2=0x7f80581036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 --2- 172.21.15.19:0/3556483185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80581032c0 0x7f80581036d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8048009900 tx=0x7f804802a410 comp rx=0 tx=0).stop 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 -- 172.21.15.19:0/3556483185 shutdown_connections 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 --2- 172.21.15.19:0/3556483185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805810d440 0x7f805810f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 --2- 172.21.15.19:0/3556483185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80581032c0 0x7f80581036d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 --2- 172.21.15.19:0/3556483185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8058107950 0x7f8058102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 -- 172.21.15.19:0/3556483185 >> 172.21.15.19:0/3556483185 conn(0x7f8058077920 msgr2=0x7f8058077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 -- 172.21.15.19:0/3556483185 shutdown_connections 2024-04-25T10:55:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 -- 172.21.15.19:0/3556483185 wait complete. 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.150+0000 7f8060384640 1 Processor -- start 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f8060384640 1 -- start start 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f8060384640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80581032c0 0x7f80581996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f8060384640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8058107950 0x7f8058199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f8060384640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f805810d440 0x7f805819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f8060384640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8058112c60 con 0x7f8058107950 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f8060384640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8058112ae0 con 0x7f80581032c0 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f8060384640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8058112de0 con 0x7f805810d440 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8058107950 0x7f8058199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8058107950 0x7f8058199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57920/0 (socket says 172.21.15.19:57920) 2024-04-25T10:55:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 -- 172.21.15.19:0/1323552156 learned_addr learned my addr 172.21.15.19:0/1323552156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805e120640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80581032c0 0x7f80581996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805e921640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f805810d440 0x7f805819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 -- 172.21.15.19:0/1323552156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f805810d440 msgr2=0x7f805819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f805810d440 0x7f805819dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 -- 172.21.15.19:0/1323552156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80581032c0 msgr2=0x7f80581996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80581032c0 0x7f80581996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805d91f640 1 -- 172.21.15.19:0/1323552156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805819e500 con 0x7f8058107950 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.151+0000 7f805e921640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f805810d440 0x7f805819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.152+0000 7f805d91f640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8058107950 0x7f8058199c00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8048032fd0 tx=0x7f8048004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.152+0000 7f804f7fe640 1 -- 172.21.15.19:0/1323552156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048046070 con 0x7f8058107950 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.152+0000 7f804f7fe640 1 -- 172.21.15.19:0/1323552156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8048003aa0 con 0x7f8058107950 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.152+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805819e790 con 0x7f8058107950 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.152+0000 7f804f7fe640 1 -- 172.21.15.19:0/1323552156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048035050 con 0x7f8058107950 2024-04-25T10:55:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.152+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8058101c70 con 0x7f8058107950 2024-04-25T10:55:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 Processor -- start 2024-04-25T10:55:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 -- start start 2024-04-25T10:55:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f804f7fe640 1 -- 172.21.15.19:0/1323552156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f8048049070 con 0x7f8058107950 2024-04-25T10:55:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e19730 0x55f2951db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f295140b00 con 0x55f294e11550 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2950dc920 con 0x55f294e19730 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f8067232740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f29513cac0 con 0x55f294e3a410 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f805e120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f805e120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951dab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57930/0 (socket says 172.21.15.19:57930) 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f805e120640 1 -- 172.21.15.19:0/3159476372 learned_addr learned my addr 172.21.15.19:0/3159476372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f805d91f640 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e19730 0x55f2951db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.155+0000 7f805e921640 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e120640 1 -- 172.21.15.19:0/3159476372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 msgr2=0x55f2951da620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e120640 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951da620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e120640 1 -- 172.21.15.19:0/3159476372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e19730 msgr2=0x55f2951db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e120640 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e19730 0x55f2951db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e120640 1 -- 172.21.15.19:0/3159476372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f294f1a7e0 con 0x55f294e11550 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e921640 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951da620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f804f7fe640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f80440afe00 0x7f80440b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e120640 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951dab60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8054009a70 tx=0x7f8054009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d01b04fffb7af98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f805e921640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f80440afe00 0x7f80440b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f804f7fe640 1 -- 172.21.15.19:0/1323552156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8048100c70 con 0x7f8058107950 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f804cbfe640 1 -- 172.21.15.19:0/3159476372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054017070 con 0x55f294e11550 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f804cbfe640 1 -- 172.21.15.19:0/3159476372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8054013470 con 0x55f294e11550 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.156+0000 7f804cbfe640 1 -- 172.21.15.19:0/3159476372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054012610 con 0x55f294e11550 2024-04-25T10:55:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 -- 172.21.15.19:0/3159476372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 msgr2=0x55f2951dab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f805e921640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f80440afe00 0x7f80440b2310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f80500095f0 tx=0x7f8050009580 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951dab60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8054009a70 tx=0x7f8054009f30 comp rx=0 tx=0).stop 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 -- 172.21.15.19:0/3159476372 shutdown_connections 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e19730 0x55f2951db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951dab60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 --2- 172.21.15.19:0/3159476372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951da620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 -- 172.21.15.19:0/3159476372 >> 172.21.15.19:0/3159476372 conn(0x55f294e2e560 msgr2=0x55f2951d6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 -- 172.21.15.19:0/3159476372 shutdown_connections 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.157+0000 7f8067232740 1 -- 172.21.15.19:0/3159476372 wait complete. 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 -- start start 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 Processor -- start 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2951e7970 con 0x55f294e19730 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2950da5d0 con 0x55f294e11550 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f2951e7af0 con 0x55f294e3a410 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f8067232740 10 client.0 _ll_register_callbacks cb 0x55f2950c8080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f805d91f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f805d91f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57932/0 (socket says 172.21.15.19:57932) 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f805d91f640 1 -- 172.21.15.19:0/1548327376 learned_addr learned my addr 172.21.15.19:0/1548327376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.158+0000 7f805e921640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805e120640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805d91f640 1 -- 172.21.15.19:0/1548327376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 msgr2=0x55f2951f0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805d91f640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951f0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805d91f640 1 -- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 msgr2=0x55f2951e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805d91f640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805d91f640 1 -- 172.21.15.19:0/1548327376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f294f7af70 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805e120640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f805d91f640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951e5120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f80480c5ed0 tx=0x7f80480c9140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4f16de5d91dc13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8041ffb640 1 -- 172.21.15.19:0/1548327376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480c4b20 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8041ffb640 1 -- 172.21.15.19:0/1548327376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f80480d1840 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8054013470 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8041ffb640 1 -- 172.21.15.19:0/1548327376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f80480c83d0 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8067232740 10 client.15531 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f294e12170 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.159+0000 7f8041ffb640 1 -- 172.21.15.19:0/1548327376 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8054013470 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8041ffb640 1 -- 172.21.15.19:0/1548327376 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f80480fe4f0 con 0x55f294e19730 2024-04-25T10:55:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8067232740 10 client.15531 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8067232740 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8067232740 20 client.15531 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8067232740 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294df91f0 0x55f2951f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8067232740 4 client.15531 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f29514e710 con 0x55f294df91f0 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f805e120640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294df91f0 0x55f2951f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f805e120640 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294df91f0 0x55f2951f6f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f254095e0d1999e0 server_cookie=c0b0e9853aa5e596 in_seq=0 out_seq=0 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.160+0000 7f8041ffb640 10 client.15531 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8041ffb640 1 -- 172.21.15.19:0/1548327376 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f29514e710 con 0x55f294df91f0 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8041ffb640 10 client.15531 handle_command_reply: tid=0 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 client.15531 shutdown 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 msgr2=0x55f2951e5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951e5120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f80480c5ed0 tx=0x7f80480c9140 comp rx=0 tx=0).stop 2024-04-25T10:55:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 shutdown_connections 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294df91f0 0x55f2951f6f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951e5120 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 --2- 172.21.15.19:0/1548327376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8041ffb640 0 client.15531 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 >> 172.21.15.19:0/1548327376 conn(0x55f294e2e560 msgr2=0x55f2951dc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 shutdown_connections 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.161+0000 7f8067232740 1 -- 172.21.15.19:0/1548327376 wait complete. 2024-04-25T10:55:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.162+0000 7f8067232740 20 client.15531 trim_cache size 0 max 16384 2024-04-25T10:55:06.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 Processor -- start 2024-04-25T10:55:06.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 -- start start 2024-04-25T10:55:06.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80581ab710 con 0x55f294e19730 2024-04-25T10:55:06.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80581adf90 con 0x55f294e11550 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f8067232740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80581abd80 con 0x55f294e3a410 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805d91f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805d91f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951fc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57946/0 (socket says 172.21.15.19:57946) 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805d91f640 1 -- 172.21.15.19:0/488328863 learned_addr learned my addr 172.21.15.19:0/488328863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805e120640 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805e921640 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805d91f640 1 -- 172.21.15.19:0/488328863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 msgr2=0x55f2951fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805d91f640 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805d91f640 1 -- 172.21.15.19:0/488328863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 msgr2=0x55f2951dc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.180+0000 7f805d91f640 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951dc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.181+0000 7f805d91f640 1 -- 172.21.15.19:0/488328863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2951e7ae0 con 0x55f294e19730 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.181+0000 7f805e120640 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.181+0000 7f805d91f640 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951fc740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f80480fe8c0 tx=0x7f80480c9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a863d26dc1b0c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.181+0000 7f8043fff640 1 -- 172.21.15.19:0/488328863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480c85b0 con 0x55f294e19730 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.181+0000 7f8043fff640 1 -- 172.21.15.19:0/488328863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8048038470 con 0x55f294e19730 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.181+0000 7f8043fff640 1 -- 172.21.15.19:0/488328863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480c8a30 con 0x55f294e19730 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 -- 172.21.15.19:0/488328863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 msgr2=0x55f2951fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951fc740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f80480fe8c0 tx=0x7f80480c9f90 comp rx=0 tx=0).stop 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 -- 172.21.15.19:0/488328863 shutdown_connections 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e3a410 0x55f2951fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e19730 0x55f2951fc740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 --2- 172.21.15.19:0/488328863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e11550 0x55f2951dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 -- 172.21.15.19:0/488328863 >> 172.21.15.19:0/488328863 conn(0x55f294e2e560 msgr2=0x55f2951dc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 -- 172.21.15.19:0/488328863 shutdown_connections 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 -- 172.21.15.19:0/488328863 wait complete. 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 -- start start 2024-04-25T10:55:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 Processor -- start 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e19730 0x55f2951ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.182+0000 7f8067232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e3a410 0x55f2951e8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8067232740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80581ac450 con 0x55f294e11550 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8067232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80581ad8c0 con 0x55f294e3a410 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8067232740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80581acb20 con 0x55f294e19730 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8067232740 10 client.0 _ll_register_callbacks cb 0x7f8048003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951ecf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57960/0 (socket says 172.21.15.19:57960) 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 -- 172.21.15.19:0/758144625 learned_addr learned my addr 172.21.15.19:0/758144625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805d91f640 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e19730 0x55f2951ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 -- 172.21.15.19:0/758144625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e19730 msgr2=0x55f2951ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e19730 0x55f2951ed450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 -- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e3a410 msgr2=0x55f2951e8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e921640 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e3a410 0x55f2951e8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e3a410 0x55f2951e8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 -- 172.21.15.19:0/758144625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2951e49f0 con 0x55f294e11550 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f805e120640 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951ecf10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8054009a70 tx=0x7f8054009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10a6ce702c07290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8041ffb640 1 -- 172.21.15.19:0/758144625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054017040 con 0x55f294e11550 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8041ffb640 1 -- 172.21.15.19:0/758144625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8054027640 con 0x55f294e11550 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2951e54b0 con 0x55f294e11550 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8041ffb640 1 -- 172.21.15.19:0/758144625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8054027840 con 0x55f294e11550 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8067232740 10 client.15537 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.183+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f29525ec50 con 0x55f294e11550 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8041ffb640 1 -- 172.21.15.19:0/758144625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f2951e54b0 con 0x55f294e11550 2024-04-25T10:55:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8041ffb640 1 -- 172.21.15.19:0/758144625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f805401c4e0 con 0x55f294e11550 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8067232740 10 client.15537 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8067232740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8067232740 20 client.15537 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8067232740 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294dce630 0x55f295261100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8067232740 4 client.15537 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f295148580 con 0x55f294dce630 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f805d91f640 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294dce630 0x55f295261100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f8041ffb640 10 client.15537 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.184+0000 7f805d91f640 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294dce630 0x55f295261100 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70425ffa6a4ea94e server_cookie=f61dde3d6d5f84de in_seq=0 out_seq=0 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8041ffb640 1 -- 172.21.15.19:0/758144625 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f295148580 con 0x55f294dce630 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8041ffb640 10 client.15537 handle_command_reply: tid=0 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 client.15537 shutdown 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 msgr2=0x55f2951ecf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951ecf10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8054009a70 tx=0x7f8054009670 comp rx=0 tx=0).stop 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 shutdown_connections 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f294dce630 0x55f295261100 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f294e3a410 0x55f2951e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f294e19730 0x55f2951ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 --2- 172.21.15.19:0/758144625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f294e11550 0x55f2951ecf10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 >> 172.21.15.19:0/758144625 conn(0x55f294e2e560 msgr2=0x55f2951dc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8041ffb640 0 client.15537 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 shutdown_connections 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 1 -- 172.21.15.19:0/758144625 wait complete. 2024-04-25T10:55:06.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.186+0000 7f8067232740 20 client.15537 trim_cache size 0 max 16384 2024-04-25T10:55:06.194 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:55:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:55:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "474407fd-8b9d-4e0e-bb2b-d35fb2b68c17", 2024-04-25T10:55:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:55:06.194 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f80440afe00 msgr2=0x7f80440b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f80440afe00 0x7f80440b2310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f80500095f0 tx=0x7f8050009580 comp rx=0 tx=0).stop 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8058107950 msgr2=0x7f8058199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8058107950 0x7f8058199c00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8048032fd0 tx=0x7f8048004270 comp rx=0 tx=0).stop 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 shutdown_connections 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f80440afe00 0x7f80440b2310 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f805810d440 0x7f805819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8058107950 0x7f8058199c00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 --2- 172.21.15.19:0/1323552156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80581032c0 0x7f80581996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 >> 172.21.15.19:0/1323552156 conn(0x7f8058077920 msgr2=0x7f80581008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 shutdown_connections 2024-04-25T10:55:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.187+0000 7f8060384640 1 -- 172.21.15.19:0/1323552156 wait complete. 2024-04-25T10:55:06.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:55:06.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:55:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3365289112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108eb0 msgr2=0x7fb8b8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3365289112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108eb0 0x7fb8b8102b50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009900 tx=0x7fb8ac02a3f0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3365289112 shutdown_connections 2024-04-25T10:55:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3365289112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b81031c0 0x7fb8b810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3365289112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108eb0 0x7fb8b8102b50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3365289112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b8108a10 0x7fb8b8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3365289112 >> 172.21.15.19:0/3365289112 conn(0x7fb8b8078110 msgr2=0x7fb8b8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3365289112 shutdown_connections 2024-04-25T10:55:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.313+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3365289112 wait complete. 2024-04-25T10:55:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 Processor -- start 2024-04-25T10:55:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 -- start start 2024-04-25T10:55:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b81031c0 0x7fb8b819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108a10 0x7fb8b819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b8108eb0 0x7fb8b819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112c10 con 0x7fb8b8108a10 2024-04-25T10:55:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112a90 con 0x7fb8b81031c0 2024-04-25T10:55:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8be3eb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112d90 con 0x7fb8b8108eb0 2024-04-25T10:55:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108a10 0x7fb8b819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108a10 0x7fb8b819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57978/0 (socket says 172.21.15.19:57978) 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.314+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3093533337 learned_addr learned my addr 172.21.15.19:0/3093533337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8bc988640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b8108eb0 0x7fb8b819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b81031c0 0x7fb8b819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3093533337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b8108eb0 msgr2=0x7fb8b819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b8108eb0 0x7fb8b819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3093533337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b81031c0 msgr2=0x7fb8b819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b81031c0 0x7fb8b819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3093533337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b819f580 con 0x7fb8b8108a10 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108a10 0x7fb8b819ad00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac032fd0 tx=0x7fb8ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b57fa640 1 -- 172.21.15.19:0/3093533337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac03f030 con 0x7fb8b8108a10 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b57fa640 1 -- 172.21.15.19:0/3093533337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ac004490 con 0x7fb8b8108a10 2024-04-25T10:55:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b819f810 con 0x7fb8b8108a10 2024-04-25T10:55:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8b57fa640 1 -- 172.21.15.19:0/3093533337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac035070 con 0x7fb8b8108a10 2024-04-25T10:55:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.315+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b819fd00 con 0x7fb8b8108a10 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 Processor -- start 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 -- start start 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8b57fa640 1 -- 172.21.15.19:0/3093533337 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7fb8ac042030 con 0x7fb8b8108a10 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b46098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b4609e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b461ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9b456fd80 con 0x55c9b4269390 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c9b45298b0 con 0x55c9b42404d0 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8c5370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c9b456bd40 con 0x55c9b42486b0 2024-04-25T10:55:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8bc988640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b46098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8bc988640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b46098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57986/0 (socket says 172.21.15.19:57986) 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8bc988640 1 -- 172.21.15.19:0/4152515738 learned_addr learned my addr 172.21.15.19:0/4152515738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b4609e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b461ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8bc988640 1 -- 172.21.15.19:0/4152515738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 msgr2=0x55c9b461ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.318+0000 7fb8bc988640 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b461ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8bc988640 1 -- 172.21.15.19:0/4152515738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 msgr2=0x55c9b4609e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8bc988640 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b4609e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8bc988640 1 -- 172.21.15.19:0/4152515738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b44faa40 con 0x55c9b4269390 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b461ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8b57fa640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb8a00afd80 0x7fb8a00b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8bc988640 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b46098f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007f40 tx=0x7fb8b000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3006be39554b1daf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb8a00afd80 0x7fb8a00b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb8b57fa640 1 -- 172.21.15.19:0/3093533337 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb8ac0ff360 con 0x7fb8b8108a10 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb89e7fc640 1 -- 172.21.15.19:0/4152515738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0010900 con 0x55c9b4269390 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb89e7fc640 1 -- 172.21.15.19:0/4152515738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8b00183f0 con 0x55c9b4269390 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.319+0000 7fb89e7fc640 1 -- 172.21.15.19:0/4152515738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00175a0 con 0x55c9b4269390 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 -- 172.21.15.19:0/4152515738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 msgr2=0x55c9b46098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb8a00afd80 0x7fb8a00b2290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c9b46072c0 tx=0x7fb8a800d500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b46098f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007f40 tx=0x7fb8b000f850 comp rx=0 tx=0).stop 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 -- 172.21.15.19:0/4152515738 shutdown_connections 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b461ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b4609e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 --2- 172.21.15.19:0/4152515738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b46098f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 -- 172.21.15.19:0/4152515738 >> 172.21.15.19:0/4152515738 conn(0x55c9b425d4e0 msgr2=0x55c9b4606210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 -- 172.21.15.19:0/4152515738 shutdown_connections 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 -- 172.21.15.19:0/4152515738 wait complete. 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.320+0000 7fb8c5370740 1 -- start start 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 1 Processor -- start 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42404d0 0x55c9b4618980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42486b0 0x55c9b4618ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b4624960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9b461b3f0 con 0x55c9b4269390 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c9b44f6d10 con 0x55c9b42486b0 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c9b461b570 con 0x55c9b42404d0 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8c5370740 10 client.0 _ll_register_callbacks cb 0x55c9b4523210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b4624960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b4624960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57990/0 (socket says 172.21.15.19:57990) 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 -- 172.21.15.19:0/1760459077 learned_addr learned my addr 172.21.15.19:0/1760459077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42486b0 0x55c9b4618ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42404d0 0x55c9b4618980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 -- 172.21.15.19:0/1760459077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42404d0 msgr2=0x55c9b4618980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42404d0 0x55c9b4618980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 -- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42486b0 msgr2=0x55c9b4618ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42486b0 0x55c9b4618ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.321+0000 7fb8bc988640 1 -- 172.21.15.19:0/1760459077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b434b7d0 con 0x55c9b4269390 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8bc988640 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b4624960 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007f10 tx=0x7fb8b0007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29257ab712a1cb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8977fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8977fe640 1 -- 172.21.15.19:0/1760459077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0025020 con 0x55c9b4269390 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8977fe640 1 -- 172.21.15.19:0/1760459077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb8b0004c50 con 0x55c9b4269390 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b00183f0 con 0x55c9b4269390 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8977fe640 1 -- 172.21.15.19:0/1760459077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb8b0030050 con 0x55c9b4269390 2024-04-25T10:55:06.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 10 client.15549 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c9b42813f0 con 0x55c9b4269390 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8977fe640 1 -- 172.21.15.19:0/1760459077 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8b00183f0 con 0x55c9b4269390 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8977fe640 1 -- 172.21.15.19:0/1760459077 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb8b0010af0 con 0x55c9b4269390 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 10 client.15549 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 20 client.15549 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b4227f60 0x55c9b462aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 4 client.15549 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.322+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9b457d990 con 0x55c9b4227f60 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.323+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b4227f60 0x55c9b462aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.323+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b4227f60 0x55c9b462aa80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e65ee0ef27a5c6 server_cookie=6c82fc8865189082 in_seq=0 out_seq=0 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.323+0000 7fb8977fe640 10 client.15549 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8977fe640 1 -- 172.21.15.19:0/1760459077 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9b457d990 con 0x55c9b4227f60 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8977fe640 10 client.15549 handle_command_reply: tid=0 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 client.15549 shutdown 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 msgr2=0x55c9b4624960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b4624960 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007f10 tx=0x7fb8b0007c40 comp rx=0 tx=0).stop 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 shutdown_connections 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b4227f60 0x55c9b462aa80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b4269390 0x55c9b4624960 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42486b0 0x55c9b4618ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 --2- 172.21.15.19:0/1760459077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42404d0 0x55c9b4618980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8977fe640 0 client.15549 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 >> 172.21.15.19:0/1760459077 conn(0x55c9b425d4e0 msgr2=0x55c9b460b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 shutdown_connections 2024-04-25T10:55:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 1 -- 172.21.15.19:0/1760459077 wait complete. 2024-04-25T10:55:06.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.324+0000 7fb8c5370740 20 client.15549 trim_cache size 0 max 16384 2024-04-25T10:55:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 Processor -- start 2024-04-25T10:55:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 -- start start 2024-04-25T10:55:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b460bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42486b0 0x55c9b46248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b4269390 0x55c9b4624e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ab7b0 con 0x55c9b42486b0 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ae010 con 0x55c9b42404d0 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8c5370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81abe00 con 0x55c9b4269390 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42486b0 0x55c9b46248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42486b0 0x55c9b46248d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58004/0 (socket says 172.21.15.19:58004) 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3862289191 learned_addr learned my addr 172.21.15.19:0/3862289191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b460bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3862289191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b4269390 msgr2=0x55c9b4624e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8bc988640 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b4269390 0x55c9b4624e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b4269390 0x55c9b4624e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3862289191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 msgr2=0x55c9b460bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b460bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 -- 172.21.15.19:0/3862289191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b42410f0 con 0x55c9b42486b0 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.343+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42486b0 0x55c9b46248d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0c4620 tx=0x7fb8ac02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ab3eb95c9c8617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb89dffb640 1 -- 172.21.15.19:0/3862289191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac049040 con 0x55c9b42486b0 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb89dffb640 1 -- 172.21.15.19:0/3862289191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ac0c6ec0 con 0x55c9b42486b0 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb89dffb640 1 -- 172.21.15.19:0/3862289191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0c3640 con 0x55c9b42486b0 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 -- 172.21.15.19:0/3862289191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42486b0 msgr2=0x55c9b46248d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42486b0 0x55c9b46248d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0c4620 tx=0x7fb8ac02a3f0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 -- 172.21.15.19:0/3862289191 shutdown_connections 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b4269390 0x55c9b4624e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42486b0 0x55c9b46248d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 --2- 172.21.15.19:0/3862289191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b42404d0 0x55c9b460bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 -- 172.21.15.19:0/3862289191 >> 172.21.15.19:0/3862289191 conn(0x55c9b425d4e0 msgr2=0x55c9b461a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 -- 172.21.15.19:0/3862289191 shutdown_connections 2024-04-25T10:55:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.344+0000 7fb8c5370740 1 -- 172.21.15.19:0/3862289191 wait complete. 2024-04-25T10:55:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 -- start start 2024-04-25T10:55:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 Processor -- start 2024-04-25T10:55:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42404d0 0x55c9b462dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b462b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b4269390 0x55c9b462b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ac4d0 con 0x55c9b42404d0 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ad940 con 0x55c9b4269390 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81acba0 con 0x55c9b42486b0 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8c5370740 10 client.0 _ll_register_callbacks cb 0x7fb8a800d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42404d0 0x55c9b462dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42404d0 0x55c9b462dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58020/0 (socket says 172.21.15.19:58020) 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.345+0000 7fb8b7fff640 1 -- 172.21.15.19:0/2842401139 learned_addr learned my addr 172.21.15.19:0/2842401139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b462b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8b7fff640 1 -- 172.21.15.19:0/2842401139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 msgr2=0x55c9b462b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b462b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8b7fff640 1 -- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b4269390 msgr2=0x55c9b462b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8bc988640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b4269390 0x55c9b462b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b4269390 0x55c9b462b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8b7fff640 1 -- 172.21.15.19:0/2842401139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b45304b0 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8bc988640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b4269390 0x55c9b462b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8b7fff640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42404d0 0x55c9b462dd50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005fd0 tx=0x7fb8a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69f593f34bec7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8977fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8977fe640 1 -- 172.21.15.19:0/2842401139 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8028620 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8977fe640 1 -- 172.21.15.19:0/2842401139 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb8a80287c0 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8ac0c6ec0 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8977fe640 1 -- 172.21.15.19:0/2842401139 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb8a8036050 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8c5370740 10 client.15555 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c9b4692660 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.346+0000 7fb8977fe640 1 -- 172.21.15.19:0/2842401139 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ac0c6ec0 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8977fe640 1 -- 172.21.15.19:0/2842401139 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb8a8019040 con 0x55c9b42404d0 2024-04-25T10:55:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8c5370740 10 client.15555 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8c5370740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8c5370740 20 client.15555 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8c5370740 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b41fd3a0 0x55c9b4694e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8c5370740 4 client.15555 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9b4577800 con 0x55c9b41fd3a0 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b41fd3a0 0x55c9b4694e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8b77fe640 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b41fd3a0 0x55c9b4694e50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0a0ef2f48ece28a server_cookie=f6106901867ded78 in_seq=0 out_seq=0 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.347+0000 7fb8977fe640 10 client.15555 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8977fe640 1 -- 172.21.15.19:0/2842401139 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c9b4577800 con 0x55c9b41fd3a0 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8977fe640 10 client.15555 handle_command_reply: tid=0 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 client.15555 shutdown 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42404d0 msgr2=0x55c9b462dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42404d0 0x55c9b462dd50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005fd0 tx=0x7fb8a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 shutdown_connections 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9b41fd3a0 0x55c9b4694e50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9b4269390 0x55c9b462b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9b42486b0 0x55c9b462b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8977fe640 0 client.15555 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 --2- 172.21.15.19:0/2842401139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9b42404d0 0x55c9b462dd50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 >> 172.21.15.19:0/2842401139 conn(0x55c9b425d4e0 msgr2=0x55c9b461a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 shutdown_connections 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 1 -- 172.21.15.19:0/2842401139 wait complete. 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8c5370740 20 client.15555 trim_cache size 0 max 16384 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:55:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "474407fd-8b9d-4e0e-bb2b-d35fb2b68c17": { 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "474407fd-8b9d-4e0e-bb2b-d35fb2b68c17", 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:55:06.356 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb8a00afd80 msgr2=0x7fb8a00b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb8a00afd80 0x7fb8a00b2290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c9b46072c0 tx=0x7fb8a800d500 comp rx=0 tx=0).stop 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108a10 msgr2=0x7fb8b819ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.348+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108a10 0x7fb8b819ad00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac032fd0 tx=0x7fb8ac004270 comp rx=0 tx=0).stop 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 shutdown_connections 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb8a00afd80 0x7fb8a00b2290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b8108eb0 0x7fb8b819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8b8108a10 0x7fb8b819ad00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 --2- 172.21.15.19:0/3093533337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b81031c0 0x7fb8b819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 >> 172.21.15.19:0/3093533337 conn(0x7fb8b8078110 msgr2=0x7fb8b8074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 shutdown_connections 2024-04-25T10:55:06.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:06.349+0000 7fb8be3eb640 1 -- 172.21.15.19:0/3093533337 wait complete. 2024-04-25T10:55:06.360 INFO:tasks.cephfs.filesystem:scrub status for tag:474407fd-8b9d-4e0e-bb2b-d35fb2b68c17 - {'path': '/', 'tag': '474407fd-8b9d-4e0e-bb2b-d35fb2b68c17', 'options': 'recursive,force'} 2024-04-25T10:55:07.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:07 smithi112 ceph-mon[32799]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-04-25T10:55:07.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:07 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:55:07.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:07 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:07.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:07 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:55:07.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:07 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:07.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:07 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:55:07.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:07 smithi019 ceph-mon[29209]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-04-25T10:55:07.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:07 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:55:07.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:07 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:07.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:07 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:55:07.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:07 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:07.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:07 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:55:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:07 smithi169 ceph-mon[33049]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-04-25T10:55:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:07 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:55:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:07 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:07 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:55:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:07 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:07 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:55:09.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:09 smithi112 ceph-mon[32799]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T10:55:09.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:09 smithi019 ceph-mon[29209]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T10:55:09.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:09 smithi169 ceph-mon[33049]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T10:55:11.303 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:11 smithi169 ceph-mon[33049]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T10:55:11.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:11 smithi112 ceph-mon[32799]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T10:55:11.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:11 smithi019 ceph-mon[29209]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T10:55:13.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:13 smithi112 ceph-mon[32799]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-25T10:55:13.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:13 smithi019 ceph-mon[29209]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-25T10:55:13.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:13 smithi169 ceph-mon[33049]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-25T10:55:15.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:15 smithi112 ceph-mon[32799]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2024-04-25T10:55:15.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:15 smithi019 ceph-mon[29209]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2024-04-25T10:55:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:15 smithi169 ceph-mon[33049]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2024-04-25T10:55:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.846+0000 7f55bd249640 1 -- 172.21.15.19:0/1179549924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81028e0 msgr2=0x7f55b8102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.846+0000 7f55bd249640 1 --2- 172.21.15.19:0/1179549924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81028e0 0x7f55b8102cf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009900 tx=0x7f55a802a3f0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 -- 172.21.15.19:0/1179549924 shutdown_connections 2024-04-25T10:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 --2- 172.21.15.19:0/1179549924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8110480 0x7f55b81128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 --2- 172.21.15.19:0/1179549924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b810db20 0x7f55b810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 --2- 172.21.15.19:0/1179549924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81028e0 0x7f55b8102cf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 -- 172.21.15.19:0/1179549924 >> 172.21.15.19:0/1179549924 conn(0x7f55b80781b0 msgr2=0x7f55b80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 -- 172.21.15.19:0/1179549924 shutdown_connections 2024-04-25T10:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 -- 172.21.15.19:0/1179549924 wait complete. 2024-04-25T10:55:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.847+0000 7f55bd249640 1 Processor -- start 2024-04-25T10:55:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55bd249640 1 -- start start 2024-04-25T10:55:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55bd249640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b81028e0 0x7f55b81a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55bd249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b810db20 0x7f55b81a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55bd249640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8110480 0x7f55b819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55bd249640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8115f40 con 0x7f55b810db20 2024-04-25T10:55:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55bd249640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8115dc0 con 0x7f55b81028e0 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55bd249640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81160c0 con 0x7f55b8110480 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b810db20 0x7f55b81a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b810db20 0x7f55b81a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49360/0 (socket says 172.21.15.19:49360) 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 -- 172.21.15.19:0/123455439 learned_addr learned my addr 172.21.15.19:0/123455439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b81028e0 0x7f55b81a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b759e640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8110480 0x7f55b819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 -- 172.21.15.19:0/123455439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8110480 msgr2=0x7f55b819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8110480 0x7f55b819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 -- 172.21.15.19:0/123455439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b81028e0 msgr2=0x7f55b81a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b81028e0 0x7f55b81a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.848+0000 7f55b659c640 1 -- 172.21.15.19:0/123455439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b819ce10 con 0x7f55b810db20 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.849+0000 7f55b659c640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b810db20 0x7f55b81a2ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f55ac005fd0 tx=0x7f55ac00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.849+0000 7f559ffff640 1 -- 172.21.15.19:0/123455439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac017070 con 0x7f55b810db20 2024-04-25T10:55:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.849+0000 7f559ffff640 1 -- 172.21.15.19:0/123455439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac009c30 con 0x7f55b810db20 2024-04-25T10:55:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.849+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b819d100 con 0x7f55b810db20 2024-04-25T10:55:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.849+0000 7f559ffff640 1 -- 172.21.15.19:0/123455439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac003d30 con 0x7f55b810db20 2024-04-25T10:55:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.849+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81a93c0 con 0x7f55b810db20 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.851+0000 7f55c41d2740 1 Processor -- start 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55c41d2740 1 -- start start 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e971b0 0x559e51236af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51237030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51237570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55c41d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e5119d460 con 0x559e50e6e2f0 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55c41d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e51157590 con 0x559e50e764d0 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55c41d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559e51199420 con 0x559e50e971b0 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51237030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51237030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49368/0 (socket says 172.21.15.19:49368) 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 -- 172.21.15.19:0/1466925958 learned_addr learned my addr 172.21.15.19:0/1466925958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b759e640 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e971b0 0x559e51236af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 -- 172.21.15.19:0/1466925958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e971b0 msgr2=0x559e51236af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e971b0 0x559e51236af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 -- 172.21.15.19:0/1466925958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 msgr2=0x559e51237570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51237570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b6d9d640 1 -- 172.21.15.19:0/1466925958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e511640d0 con 0x559e50e6e2f0 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f55b759e640 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e971b0 0x559e51236af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.852+0000 7f559ffff640 1 -- 172.21.15.19:0/123455439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f55ac003ed0 con 0x7f55b810db20 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.853+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51237030 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009900 tx=0x7f55a8032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ef75ba8e771c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.853+0000 7f559d7fa640 1 -- 172.21.15.19:0/1466925958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8040070 con 0x559e50e6e2f0 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.853+0000 7f559d7fa640 1 -- 172.21.15.19:0/1466925958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a8005db0 con 0x559e50e6e2f0 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.853+0000 7f559d7fa640 1 -- 172.21.15.19:0/1466925958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8035040 con 0x559e50e6e2f0 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.853+0000 7f559ffff640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f55a00afd70 0x7f55a00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.853+0000 7f55b759e640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f55a00afd70 0x7f55a00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.853+0000 7f559ffff640 1 -- 172.21.15.19:0/123455439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f55ac0def80 con 0x7f55b810db20 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 -- 172.21.15.19:0/1466925958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 msgr2=0x559e51237030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51237030 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009900 tx=0x7f55a8032db0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 -- 172.21.15.19:0/1466925958 shutdown_connections 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51237570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51237030 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 --2- 172.21.15.19:0/1466925958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e971b0 0x559e51236af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 -- 172.21.15.19:0/1466925958 >> 172.21.15.19:0/1466925958 conn(0x559e50e8b300 msgr2=0x559e512332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 -- 172.21.15.19:0/1466925958 shutdown_connections 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55b759e640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f55a00afd70 0x7f55a00b2280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x559e51233c60 tx=0x7f55b000a3e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 -- 172.21.15.19:0/1466925958 wait complete. 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 -- start start 2024-04-25T10:55:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.854+0000 7f55c41d2740 1 Processor -- start 2024-04-25T10:55:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e6e2f0 0x559e512411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51241750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e971b0 0x559e51247110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55c41d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e51243bc0 con 0x559e50e971b0 2024-04-25T10:55:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55c41d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e51124b60 con 0x559e50e764d0 2024-04-25T10:55:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55c41d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559e51243d40 con 0x559e50e6e2f0 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55c41d2740 10 client.0 _ll_register_callbacks cb 0x559e511492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e971b0 0x559e51247110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e971b0 0x559e51247110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49380/0 (socket says 172.21.15.19:49380) 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 -- 172.21.15.19:0/3478144265 learned_addr learned my addr 172.21.15.19:0/3478144265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e6e2f0 0x559e512411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b659c640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51241750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 -- 172.21.15.19:0/3478144265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e6e2f0 msgr2=0x559e512411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e6e2f0 0x559e512411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 -- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 msgr2=0x559e51241750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51241750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 -- 172.21.15.19:0/3478144265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e50eaf210 con 0x559e50e971b0 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b659c640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51241750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e6e2f0 0x559e512411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f55b759e640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e971b0 0x559e51247110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f55b00119f0 tx=0x7f55b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924ca99804f42f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.855+0000 7f559aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f559aff5640 1 -- 172.21.15.19:0/3478144265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0022070 con 0x559e50e971b0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f559aff5640 1 -- 172.21.15.19:0/3478144265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f55b0010d60 con 0x559e50e971b0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a8005db0 con 0x559e50e971b0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f559aff5640 1 -- 172.21.15.19:0/3478144265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f55b002e050 con 0x559e50e971b0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 10 client.15567 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559e50f7b620 con 0x559e50e971b0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f559aff5640 1 -- 172.21.15.19:0/3478144265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a8005db0 con 0x559e50e971b0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f559aff5640 1 -- 172.21.15.19:0/3478144265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f55b00263b0 con 0x559e50e971b0 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 10 client.15567 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 20 client.15567 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e55d80 0x559e51253760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 4 client.15567 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.856+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e511ab070 con 0x559e50e55d80 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.857+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e55d80 0x559e51253760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.857+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e55d80 0x559e51253760 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e000210950f4df81 server_cookie=e8d8a16129370a04 in_seq=0 out_seq=0 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.857+0000 7f559aff5640 10 client.15567 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f559aff5640 1 -- 172.21.15.19:0/3478144265 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e511ab070 con 0x559e50e55d80 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f559aff5640 10 client.15567 handle_command_reply: tid=0 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 client.15567 shutdown 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e971b0 msgr2=0x559e51247110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e971b0 0x559e51247110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f55b00119f0 tx=0x7f55b0004ab0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 shutdown_connections 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e55d80 0x559e51253760 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e971b0 0x559e51247110 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e764d0 0x559e51241750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f559aff5640 0 client.15567 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3478144265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e6e2f0 0x559e512411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 >> 172.21.15.19:0/3478144265 conn(0x559e50e8b300 msgr2=0x559e51238810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 shutdown_connections 2024-04-25T10:55:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 1 -- 172.21.15.19:0/3478144265 wait complete. 2024-04-25T10:55:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.858+0000 7f55c41d2740 20 client.15567 trim_cache size 0 max 16384 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.875+0000 7f55c41d2740 1 Processor -- start 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.875+0000 7f55c41d2740 1 -- start start 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51247010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51258f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 0x559e512594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55c41d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81afad0 con 0x559e50e6e2f0 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55c41d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81b2350 con 0x559e50e55d80 2024-04-25T10:55:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55c41d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81b0140 con 0x559e50e764d0 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51258f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51258f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49388/0 (socket says 172.21.15.19:49388) 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 -- 172.21.15.19:0/3713677971 learned_addr learned my addr 172.21.15.19:0/3713677971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 -- 172.21.15.19:0/3713677971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 msgr2=0x559e512594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 --2- 172.21.15.19:0/3713677971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 0x559e512594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 -- 172.21.15.19:0/3713677971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 msgr2=0x559e51247010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 --2- 172.21.15.19:0/3713677971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51247010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 -- 172.21.15.19:0/3713677971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e51247550 con 0x559e50e6e2f0 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f55b659c640 1 --2- 172.21.15.19:0/3713677971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51258f60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0a6c00 tx=0x7f55ac0a6980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277f324d01829d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f559cff9640 1 -- 172.21.15.19:0/3713677971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0ab2b0 con 0x559e50e6e2f0 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f559cff9640 1 -- 172.21.15.19:0/3713677971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac0dce50 con 0x559e50e6e2f0 2024-04-25T10:55:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.876+0000 7f559cff9640 1 -- 172.21.15.19:0/3713677971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0041c0 con 0x559e50e6e2f0 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.877+0000 7f55c41d2740 1 -- 172.21.15.19:0/3713677971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 msgr2=0x559e51258f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.877+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3713677971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51258f60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0a6c00 tx=0x7f55ac0a6980 comp rx=0 tx=0).stop 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.877+0000 7f55c41d2740 1 -- 172.21.15.19:0/3713677971 shutdown_connections 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.877+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3713677971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 0x559e512594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.877+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3713677971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51258f60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.877+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3713677971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51247010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 -- 172.21.15.19:0/3713677971 >> 172.21.15.19:0/3713677971 conn(0x559e50e8b300 msgr2=0x559e5123f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 -- 172.21.15.19:0/3713677971 shutdown_connections 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 -- 172.21.15.19:0/3713677971 wait complete. 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 -- start start 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 Processor -- start 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51248ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51249420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 0x559e51249960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81b0810 con 0x559e50e6e2f0 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81b1c80 con 0x559e50e55d80 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81b0ee0 con 0x559e50e764d0 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55c41d2740 10 client.0 _ll_register_callbacks cb 0x7f55b0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.878+0000 7f55b659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51249420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51249420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49400/0 (socket says 172.21.15.19:49400) 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 -- 172.21.15.19:0/3755417809 learned_addr learned my addr 172.21.15.19:0/3755417809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51248ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b759e640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 0x559e51249960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 -- 172.21.15.19:0/3755417809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 msgr2=0x559e51249960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 0x559e51249960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 -- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 msgr2=0x559e51248ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51248ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 -- 172.21.15.19:0/3755417809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5124def0 con 0x559e50e6e2f0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51248ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55b659c640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51249420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0dcd00 tx=0x7f55ac0afdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaafa962ba7820d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f559aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f559aff5640 1 -- 172.21.15.19:0/3755417809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac06b070 con 0x559e50e6e2f0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f559aff5640 1 -- 172.21.15.19:0/3755417809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f55ac0041c0 con 0x559e50e6e2f0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ac0dce50 con 0x559e50e6e2f0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f559aff5640 1 -- 172.21.15.19:0/3755417809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f55ac0ab2b0 con 0x559e50e6e2f0 2024-04-25T10:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55c41d2740 10 client.15573 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559e512bbbb0 con 0x559e50e6e2f0 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.879+0000 7f559aff5640 1 -- 172.21.15.19:0/3755417809 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac0dce50 con 0x559e50e6e2f0 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f559aff5640 1 -- 172.21.15.19:0/3755417809 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f55ac0a5c60 con 0x559e50e6e2f0 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55c41d2740 10 client.15573 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55c41d2740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55c41d2740 20 client.15573 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e971b0 0x559e512be2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55c41d2740 4 client.15573 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e511a4ee0 con 0x559e50e971b0 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e971b0 0x559e512be2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f55b6d9d640 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e971b0 0x559e512be2d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f0cabe5d5bd2a9 server_cookie=b40e4206d7eb70e8 in_seq=0 out_seq=0 2024-04-25T10:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.880+0000 7f559aff5640 10 client.15573 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f559aff5640 1 -- 172.21.15.19:0/3755417809 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e511a4ee0 con 0x559e50e971b0 2024-04-25T10:55:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f559aff5640 10 client.15573 handle_command_reply: tid=0 2024-04-25T10:55:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 client.15573 shutdown 2024-04-25T10:55:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 msgr2=0x559e51249420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51249420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0dcd00 tx=0x7f55ac0afdd0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 shutdown_connections 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e50e971b0 0x559e512be2d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e50e764d0 0x559e51249960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e50e6e2f0 0x559e51249420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f559aff5640 0 client.15573 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 --2- 172.21.15.19:0/3755417809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e50e55d80 0x559e51248ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 >> 172.21.15.19:0/3755417809 conn(0x559e50e8b300 msgr2=0x559e51236880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 shutdown_connections 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.888+0000 7f55c41d2740 1 -- 172.21.15.19:0/3755417809 wait complete. 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55c41d2740 20 client.15573 trim_cache size 0 max 16384 2024-04-25T10:55:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:55:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:55:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:55:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f55a00afd70 msgr2=0x7f55a00b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f55a00afd70 0x7f55a00b2280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x559e51233c60 tx=0x7f55b000a3e0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b810db20 msgr2=0x7f55b81a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b810db20 0x7f55b81a2ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f55ac005fd0 tx=0x7f55ac00dde0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 shutdown_connections 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f55a00afd70 0x7f55a00b2280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8110480 0x7f55b819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b810db20 0x7f55b81a2ab0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 --2- 172.21.15.19:0/123455439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b81028e0 0x7f55b81a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 >> 172.21.15.19:0/123455439 conn(0x7f55b80781b0 msgr2=0x7f55b8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 shutdown_connections 2024-04-25T10:55:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:15.889+0000 7f55bd249640 1 -- 172.21.15.19:0/123455439 wait complete. 2024-04-25T10:55:17.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:17 smithi112 ceph-mon[32799]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-25T10:55:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:17 smithi019 ceph-mon[29209]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-25T10:55:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:17 smithi169 ceph-mon[33049]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-25T10:55:19.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:19 smithi112 ceph-mon[32799]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:55:19.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:19 smithi019 ceph-mon[29209]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:55:19.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:19 smithi169 ceph-mon[33049]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:55:21.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:21 smithi112 ceph-mon[32799]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-25T10:55:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:21 smithi019 ceph-mon[29209]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-25T10:55:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:21 smithi169 ceph-mon[33049]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-25T10:55:23.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:23 smithi112 ceph-mon[32799]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-25T10:55:23.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:23 smithi019 ceph-mon[29209]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-25T10:55:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:23 smithi169 ceph-mon[33049]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-25T10:55:25.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:25 smithi019 ceph-mon[29209]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-25T10:55:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:25 smithi169 ceph-mon[33049]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-25T10:55:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:25 smithi112 ceph-mon[32799]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-25T10:55:27.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:27 smithi019 ceph-mon[29209]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-25T10:55:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:27 smithi169 ceph-mon[33049]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-25T10:55:27.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:27 smithi112 ceph-mon[32799]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-25T10:55:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 -- 172.21.15.19:0/1274366489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8810e0e0 msgr2=0x7fee881105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 --2- 172.21.15.19:0/1274366489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8810e0e0 0x7fee881105d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fee80009ca0 tx=0x7fee8002a9e0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 -- 172.21.15.19:0/1274366489 shutdown_connections 2024-04-25T10:55:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 --2- 172.21.15.19:0/1274366489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8810e0e0 0x7fee881105d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 --2- 172.21.15.19:0/1274366489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee8810b730 0x7fee8810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 --2- 172.21.15.19:0/1274366489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee88104cd0 0x7fee881050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 -- 172.21.15.19:0/1274366489 >> 172.21.15.19:0/1274366489 conn(0x7fee880781b0 msgr2=0x7fee880785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 -- 172.21.15.19:0/1274366489 shutdown_connections 2024-04-25T10:55:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.011+0000 7fee8df46640 1 -- 172.21.15.19:0/1274366489 wait complete. 2024-04-25T10:55:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.012+0000 7fee8df46640 1 Processor -- start 2024-04-25T10:55:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.012+0000 7fee8df46640 1 -- start start 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.012+0000 7fee8df46640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88104cd0 0x7fee8819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.012+0000 7fee8df46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee8810b730 0x7fee8819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee8df46640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee8810e0e0 0x7fee8819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee8df46640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee88113d40 con 0x7fee88104cd0 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee8df46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee88113bc0 con 0x7fee8810b730 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee8df46640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee88113ec0 con 0x7fee8810e0e0 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88104cd0 0x7fee8819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88104cd0 0x7fee8819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56286/0 (socket says 172.21.15.19:56286) 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 -- 172.21.15.19:0/3779126574 learned_addr learned my addr 172.21.15.19:0/3779126574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee87fff640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee8810e0e0 0x7fee8819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee86ffd640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee8810b730 0x7fee8819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 -- 172.21.15.19:0/3779126574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee8810e0e0 msgr2=0x7fee8819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee8810e0e0 0x7fee8819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 -- 172.21.15.19:0/3779126574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee8810b730 msgr2=0x7fee8819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee8810b730 0x7fee8819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 -- 172.21.15.19:0/3779126574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8819f560 con 0x7fee88104cd0 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee87fff640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee8810e0e0 0x7fee8819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.013+0000 7fee877fe640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88104cd0 0x7fee8819a700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee7800d950 tx=0x7fee7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.014+0000 7fee84ff9640 1 -- 172.21.15.19:0/3779126574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78017030 con 0x7fee88104cd0 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.014+0000 7fee84ff9640 1 -- 172.21.15.19:0/3779126574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78004020 con 0x7fee88104cd0 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.014+0000 7fee84ff9640 1 -- 172.21.15.19:0/3779126574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780047a0 con 0x7fee88104cd0 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.014+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8819f850 con 0x7fee88104cd0 2024-04-25T10:55:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.014+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8819fd50 con 0x7fee88104cd0 2024-04-25T10:55:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 Processor -- start 2024-04-25T10:55:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 -- start start 2024-04-25T10:55:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67bbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67bc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557de6722460 con 0x557de63f32f0 2024-04-25T10:55:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557de66dc590 con 0x557de641c1b0 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.017+0000 7fee94dfc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557de671e420 con 0x557de63fb4d0 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67bbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67bbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:35760/0 (socket says 172.21.15.19:35760) 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee84ff9640 1 -- 172.21.15.19:0/3779126574 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7fee78009a30 con 0x7fee88104cd0 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 -- 172.21.15.19:0/411257490 learned_addr learned my addr 172.21.15.19:0/411257490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee86ffd640 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67bc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee877fe640 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 -- 172.21.15.19:0/411257490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 msgr2=0x557de67bc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67bc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 -- 172.21.15.19:0/411257490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 msgr2=0x557de67bc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67bc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 -- 172.21.15.19:0/411257490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de66e90d0 con 0x557de641c1b0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee877fe640 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.018+0000 7fee87fff640 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67bbaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee80009880 tx=0x7fee800099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb5c14a4e756b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.019+0000 7fee6dffb640 1 -- 172.21.15.19:0/411257490 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80047070 con 0x557de641c1b0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.019+0000 7fee6dffb640 1 -- 172.21.15.19:0/411257490 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee80003e20 con 0x557de641c1b0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.019+0000 7fee6dffb640 1 -- 172.21.15.19:0/411257490 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80033040 con 0x557de641c1b0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.019+0000 7fee84ff9640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee700afdf0 0x7fee700b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.019+0000 7fee877fe640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee700afdf0 0x7fee700b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.019+0000 7fee84ff9640 1 -- 172.21.15.19:0/3779126574 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fee780ded30 con 0x7fee88104cd0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee877fe640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee700afdf0 0x7fee700b2300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee780dc390 tx=0x7fee780046c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 -- 172.21.15.19:0/411257490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 msgr2=0x557de67bbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67bbaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee80009880 tx=0x7fee800099f0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 -- 172.21.15.19:0/411257490 shutdown_connections 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67bc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 --2- 172.21.15.19:0/411257490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67bbaf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 -- 172.21.15.19:0/411257490 >> 172.21.15.19:0/411257490 conn(0x557de6410300 msgr2=0x557de67b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 -- 172.21.15.19:0/411257490 shutdown_connections 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 -- 172.21.15.19:0/411257490 wait complete. 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.020+0000 7fee94dfc740 1 -- start start 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 1 Processor -- start 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557de67c8bc0 con 0x557de63f32f0 2024-04-25T10:55:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557de66a9b60 con 0x557de641c1b0 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557de67c8d40 con 0x557de63fb4d0 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee94dfc740 10 client.0 _ll_register_callbacks cb 0x557de66ce2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56304/0 (socket says 172.21.15.19:56304) 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.021+0000 7fee87fff640 1 -- 172.21.15.19:0/1509549414 learned_addr learned my addr 172.21.15.19:0/1509549414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee86ffd640 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee87fff640 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee87fff640 1 -- 172.21.15.19:0/1509549414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 msgr2=0x557de67c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee87fff640 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee87fff640 1 -- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 msgr2=0x557de67cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee87fff640 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee87fff640 1 -- 172.21.15.19:0/1509549414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de6434210 con 0x557de63f32f0 2024-04-25T10:55:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee87fff640 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67c61b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fee80005cc0 tx=0x7fee8002e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bf9795bdf5035a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee66ffd640 1 -- 172.21.15.19:0/1509549414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80052060 con 0x557de63f32f0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee80003e20 con 0x557de63f32f0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.022+0000 7fee66ffd640 1 -- 172.21.15.19:0/1509549414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fee8002f850 con 0x557de63f32f0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee66ffd640 1 -- 172.21.15.19:0/1509549414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fee80033070 con 0x557de63f32f0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 10 client.15582 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557de6500620 con 0x557de63f32f0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee66ffd640 1 -- 172.21.15.19:0/1509549414 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee80003e20 con 0x557de63f32f0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee66ffd640 1 -- 172.21.15.19:0/1509549414 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fee80004520 con 0x557de63f32f0 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 10 client.15582 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 20 client.15582 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de63dad80 0x557de67d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 4 client.15582 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.023+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557de6730070 con 0x557de63dad80 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.024+0000 7fee86ffd640 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de63dad80 0x557de67d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.024+0000 7fee66ffd640 10 client.15582 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.024+0000 7fee86ffd640 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de63dad80 0x557de67d84c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36fd739d176d7e16 server_cookie=da9bf7c780e426e5 in_seq=0 out_seq=0 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee66ffd640 1 -- 172.21.15.19:0/1509549414 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557de6730070 con 0x557de63dad80 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee66ffd640 10 client.15582 handle_command_reply: tid=0 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 client.15582 shutdown 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 msgr2=0x557de67c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67c61b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fee80005cc0 tx=0x7fee8002e6d0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 shutdown_connections 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de63dad80 0x557de67d84c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de641c1b0 0x557de67cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee66ffd640 0 client.15582 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63fb4d0 0x557de67c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1509549414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63f32f0 0x557de67c61b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.025+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 >> 172.21.15.19:0/1509549414 conn(0x557de6410300 msgr2=0x557de67bb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.026+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 shutdown_connections 2024-04-25T10:55:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.026+0000 7fee94dfc740 1 -- 172.21.15.19:0/1509549414 wait complete. 2024-04-25T10:55:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.026+0000 7fee94dfc740 20 client.15582 trim_cache size 0 max 16384 2024-04-25T10:55:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 Processor -- start 2024-04-25T10:55:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 -- start start 2024-04-25T10:55:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63dad80 0x557de67c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63f32f0 0x557de67cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63fb4d0 0x557de67c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ab7f0 con 0x557de63dad80 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ae050 con 0x557de63fb4d0 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee94dfc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee881abe40 con 0x557de63f32f0 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63dad80 0x557de67c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63dad80 0x557de67c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56316/0 (socket says 172.21.15.19:56316) 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.056+0000 7fee87fff640 1 -- 172.21.15.19:0/2293464523 learned_addr learned my addr 172.21.15.19:0/2293464523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee87fff640 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63fb4d0 0x557de67c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee86ffd640 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63f32f0 0x557de67cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee87fff640 1 -- 172.21.15.19:0/2293464523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63f32f0 msgr2=0x557de67cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee87fff640 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63f32f0 0x557de67cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee87fff640 1 -- 172.21.15.19:0/2293464523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63fb4d0 msgr2=0x557de67c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee87fff640 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63fb4d0 0x557de67c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee87fff640 1 -- 172.21.15.19:0/2293464523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de67b8780 con 0x557de63dad80 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee86ffd640 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63f32f0 0x557de67cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee87fff640 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63dad80 0x557de67c8470 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fee80009880 tx=0x7fee8002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2555741b85b2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee6d7fa640 1 -- 172.21.15.19:0/2293464523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80052040 con 0x557de63dad80 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee6d7fa640 1 -- 172.21.15.19:0/2293464523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8002fd00 con 0x557de63dad80 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.057+0000 7fee6d7fa640 1 -- 172.21.15.19:0/2293464523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80047040 con 0x557de63dad80 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 -- 172.21.15.19:0/2293464523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63dad80 msgr2=0x557de67c8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63dad80 0x557de67c8470 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fee80009880 tx=0x7fee8002a9e0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 -- 172.21.15.19:0/2293464523 shutdown_connections 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63fb4d0 0x557de67c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63f32f0 0x557de67cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 --2- 172.21.15.19:0/2293464523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63dad80 0x557de67c8470 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 -- 172.21.15.19:0/2293464523 >> 172.21.15.19:0/2293464523 conn(0x557de6410300 msgr2=0x557de67c4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 -- 172.21.15.19:0/2293464523 shutdown_connections 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 -- 172.21.15.19:0/2293464523 wait complete. 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.058+0000 7fee94dfc740 1 -- start start 2024-04-25T10:55:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 1 Processor -- start 2024-04-25T10:55:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63dad80 0x557de67d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63f32f0 0x557de67cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63fb4d0 0x557de67cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ac510 con 0x557de63fb4d0 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ad980 con 0x557de63f32f0 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee881acbe0 con 0x557de63dad80 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee94dfc740 10 client.0 _ll_register_callbacks cb 0x7fee80045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63fb4d0 0x557de67cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63fb4d0 0x557de67cfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56332/0 (socket says 172.21.15.19:56332) 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee87fff640 1 -- 172.21.15.19:0/1620508432 learned_addr learned my addr 172.21.15.19:0/1620508432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.059+0000 7fee87fff640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63dad80 0x557de67d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee86ffd640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63f32f0 0x557de67cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee87fff640 1 -- 172.21.15.19:0/1620508432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63dad80 msgr2=0x557de67d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee87fff640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63dad80 0x557de67d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee87fff640 1 -- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63f32f0 msgr2=0x557de67cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee87fff640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63f32f0 0x557de67cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee87fff640 1 -- 172.21.15.19:0/1620508432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de67d0780 con 0x557de63fb4d0 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee86ffd640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63f32f0 0x557de67cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee87fff640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63fb4d0 0x557de67cfff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee800098d0 tx=0x7fee8003b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad1a69da8de364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee66ffd640 1 -- 172.21.15.19:0/1620508432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80052070 con 0x557de63fb4d0 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee66ffd640 1 -- 172.21.15.19:0/1620508432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fee80038a90 con 0x557de63fb4d0 2024-04-25T10:55:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8002fd00 con 0x557de63fb4d0 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.060+0000 7fee66ffd640 1 -- 172.21.15.19:0/1620508432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fee80047070 con 0x557de63fb4d0 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 10 client.15588 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557de68404f0 con 0x557de63fb4d0 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee66ffd640 1 -- 172.21.15.19:0/1620508432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8002fd00 con 0x557de63fb4d0 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee66ffd640 1 -- 172.21.15.19:0/1620508432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fee80004350 con 0x557de63fb4d0 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 10 client.15588 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 20 client.15588 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de641c1b0 0x557de6842980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 4 client.15588 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:55:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.061+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557de6729ee0 con 0x557de641c1b0 2024-04-25T10:55:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.062+0000 7fee86ffd640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de641c1b0 0x557de6842980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.062+0000 7fee66ffd640 10 client.15588 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.062+0000 7fee86ffd640 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de641c1b0 0x557de6842980 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e4a983e4d7df23 server_cookie=d1109e5cefdac4d2 in_seq=0 out_seq=0 2024-04-25T10:55:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee66ffd640 1 -- 172.21.15.19:0/1620508432 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557de6729ee0 con 0x557de641c1b0 2024-04-25T10:55:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee66ffd640 10 client.15588 handle_command_reply: tid=0 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 client.15588 shutdown 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63fb4d0 msgr2=0x557de67cfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63fb4d0 0x557de67cfff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee800098d0 tx=0x7fee8003b040 comp rx=0 tx=0).stop 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 shutdown_connections 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557de641c1b0 0x557de6842980 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557de63fb4d0 0x557de67cfff0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee66ffd640 0 client.15588 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557de63f32f0 0x557de67cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 --2- 172.21.15.19:0/1620508432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557de63dad80 0x557de67d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 >> 172.21.15.19:0/1620508432 conn(0x557de6410300 msgr2=0x557de67c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 shutdown_connections 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 1 -- 172.21.15.19:0/1620508432 wait complete. 2024-04-25T10:55:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.067+0000 7fee94dfc740 20 client.15588 trim_cache size 0 max 16384 2024-04-25T10:55:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:55:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:55:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:55:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:55:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee700afdf0 msgr2=0x7fee700b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee700afdf0 0x7fee700b2300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee780dc390 tx=0x7fee780046c0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88104cd0 msgr2=0x7fee8819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88104cd0 0x7fee8819a700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee7800d950 tx=0x7fee7800de10 comp rx=0 tx=0).stop 2024-04-25T10:55:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 shutdown_connections 2024-04-25T10:55:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee700afdf0 0x7fee700b2300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee8810e0e0 0x7fee8819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee8810b730 0x7fee8819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 --2- 172.21.15.19:0/3779126574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88104cd0 0x7fee8819a700 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.068+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 >> 172.21.15.19:0/3779126574 conn(0x7fee880781b0 msgr2=0x7fee88101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.069+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 shutdown_connections 2024-04-25T10:55:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:29.069+0000 7fee8df46640 1 -- 172.21.15.19:0/3779126574 wait complete. 2024-04-25T10:55:29.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:29 smithi019 ceph-mon[29209]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 682 B/s wr, 3 op/s 2024-04-25T10:55:29.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:29 smithi169 ceph-mon[33049]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 682 B/s wr, 3 op/s 2024-04-25T10:55:29.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:29 smithi112 ceph-mon[32799]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 682 B/s wr, 3 op/s 2024-04-25T10:55:31.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:31 smithi019 ceph-mon[29209]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T10:55:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:31 smithi169 ceph-mon[33049]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T10:55:31.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:31 smithi112 ceph-mon[32799]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T10:55:33.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:33 smithi019 ceph-mon[29209]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:33 smithi169 ceph-mon[33049]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:33 smithi112 ceph-mon[32799]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:35.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:35 smithi019 ceph-mon[29209]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:35.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:35 smithi169 ceph-mon[33049]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:35.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:35 smithi112 ceph-mon[32799]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:36.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:36 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:55:36.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:36 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:36.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:36 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:36.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:55:36.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:55:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/333339232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 msgr2=0x7fc2d010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/333339232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d010f540 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009ca0 tx=0x7fc2c802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/333339232 shutdown_connections 2024-04-25T10:55:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/333339232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d010f540 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/333339232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d0108ec0 0x7fc2d01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/333339232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2d0108a20 0x7fc2d0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/333339232 >> 172.21.15.19:0/333339232 conn(0x7fc2d0077880 msgr2=0x7fc2d0077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:36.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/333339232 shutdown_connections 2024-04-25T10:55:36.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.495+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/333339232 wait complete. 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.496+0000 7fc2d5f4d640 1 Processor -- start 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2d5f4d640 1 -- start start 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2d5f4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d0199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2d5f4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d0108a20 0x7fc2d01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2d5f4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2d0108ec0 0x7fc2d019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2d5f4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0112c60 con 0x7fc2d0103830 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2d5f4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0112ae0 con 0x7fc2d0108ec0 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2d5f4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0112de0 con 0x7fc2d0108a20 2024-04-25T10:55:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d0199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d0199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51012/0 (socket says 172.21.15.19:51012) 2024-04-25T10:55:36.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.497+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/2780975944 learned_addr learned my addr 172.21.15.19:0/2780975944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:36.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d0108a20 0x7fc2d01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cffff640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2d0108ec0 0x7fc2d019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/2780975944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d0108a20 msgr2=0x7fc2d01999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d0108a20 0x7fc2d01999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/2780975944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2d0108ec0 msgr2=0x7fc2d019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2d0108ec0 0x7fc2d019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/2780975944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d019e2c0 con 0x7fc2d0103830 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cffff640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2d0108ec0 0x7fc2d019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.498+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d0199460 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e9f0 tx=0x7fc2c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.499+0000 7fc2ccff9640 1 -- 172.21.15.19:0/2780975944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000cd80 con 0x7fc2d0103830 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.499+0000 7fc2ccff9640 1 -- 172.21.15.19:0/2780975944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c00108e0 con 0x7fc2d0103830 2024-04-25T10:55:36.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.499+0000 7fc2ccff9640 1 -- 172.21.15.19:0/2780975944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0018980 con 0x7fc2d0103830 2024-04-25T10:55:36.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.499+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d019e5b0 con 0x7fc2d0103830 2024-04-25T10:55:36.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.499+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d019eab0 con 0x7fc2d0103830 2024-04-25T10:55:36.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:36 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:55:36.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:36 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:36.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:36 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.502+0000 7fc2dce03740 1 Processor -- start 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2dce03740 1 -- start start 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b20e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df6175f6b0 0x55df61b21350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2dce03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df61a86d80 con 0x55df617574d0 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2dce03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df61a408b0 con 0x55df61780390 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2dce03740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df61a82d40 con 0x55df6175f6b0 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b20e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2ccff9640 1 -- 172.21.15.19:0/2780975944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7fc2c0010430 con 0x7fc2d0103830 2024-04-25T10:55:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b20e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51014/0 (socket says 172.21.15.19:51014) 2024-04-25T10:55:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/943919598 learned_addr learned my addr 172.21.15.19:0/943919598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.503+0000 7fc2cffff640 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df6175f6b0 0x55df61b21350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/943919598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df6175f6b0 msgr2=0x55df61b21350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df6175f6b0 0x55df61b21350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/943919598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 msgr2=0x55df61b208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/943919598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df61a11a40 con 0x55df617574d0 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b20e10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00dcde0 tx=0x7fc2c00e7200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4654e93c47e66e15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.504+0000 7fc2ccff9640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc2b80afe70 0x7fc2b80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.505+0000 7fc2ccff9640 1 -- 172.21.15.19:0/2780975944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc2c00df8a0 con 0x7fc2d0103830 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.505+0000 7fc2cffff640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc2b80afe70 0x7fc2b80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.505+0000 7fc2b5ffb640 1 -- 172.21.15.19:0/943919598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00a3680 con 0x55df617574d0 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.505+0000 7fc2b5ffb640 1 -- 172.21.15.19:0/943919598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c00f0070 con 0x55df617574d0 2024-04-25T10:55:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.505+0000 7fc2b5ffb640 1 -- 172.21.15.19:0/943919598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00a6f30 con 0x55df617574d0 2024-04-25T10:55:36.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 -- 172.21.15.19:0/943919598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 msgr2=0x55df61b20e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b20e10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00dcde0 tx=0x7fc2c00e7200 comp rx=0 tx=0).stop 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2cffff640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc2b80afe70 0x7fc2b80b2380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55df61b1db60 tx=0x7fc2c80095a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 -- 172.21.15.19:0/943919598 shutdown_connections 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df6175f6b0 0x55df61b21350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b20e10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 --2- 172.21.15.19:0/943919598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 -- 172.21.15.19:0/943919598 >> 172.21.15.19:0/943919598 conn(0x55df617744e0 msgr2=0x55df61b1d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 -- 172.21.15.19:0/943919598 shutdown_connections 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.506+0000 7fc2dce03740 1 -- 172.21.15.19:0/943919598 wait complete. 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.507+0000 7fc2dce03740 1 -- start start 2024-04-25T10:55:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.507+0000 7fc2dce03740 1 Processor -- start 2024-04-25T10:55:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b30590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b30af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b2b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b30590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b30590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51020/0 (socket says 172.21.15.19:51020) 2024-04-25T10:55:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2dce03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df61b27f90 con 0x55df617574d0 2024-04-25T10:55:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2dce03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df61a0dd10 con 0x55df6175f6b0 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2dce03740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df61b28110 con 0x55df61780390 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b30af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b30af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:39944/0 (socket says 172.21.15.19:39944) 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 -- 172.21.15.19:0/2377101220 learned_addr learned my addr 172.21.15.19:0/2377101220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2dce03740 10 client.0 _ll_register_callbacks cb 0x55df61a3a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2cffff640 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b2b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 -- 172.21.15.19:0/2377101220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 msgr2=0x55df61b2b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b2b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 -- 172.21.15.19:0/2377101220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 msgr2=0x55df61b30590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b30590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2ceffd640 1 -- 172.21.15.19:0/2377101220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df618627d0 con 0x55df6175f6b0 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.508+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b30590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b30af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4012730 tx=0x7fc2c4012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a63c095cc62da7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2aeffd640 1 -- 172.21.15.19:0/2377101220 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4004050 con 0x55df6175f6b0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c00f0070 con 0x55df6175f6b0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2aeffd640 1 -- 172.21.15.19:0/2377101220 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc2c40041f0 con 0x55df6175f6b0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2aeffd640 1 -- 172.21.15.19:0/2377101220 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc2c40043f0 con 0x55df6175f6b0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2dce03740 10 client.25292 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.509+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55df617983f0 con 0x55df6175f6b0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2aeffd640 1 -- 172.21.15.19:0/2377101220 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c400fdd0 con 0x55df6175f6b0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2aeffd640 1 -- 172.21.15.19:0/2377101220 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc2c4010430 con 0x55df6175f6b0 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2dce03740 10 client.25292 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2dce03740 10 client.25292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2dce03740 20 client.25292 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2dce03740 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df6173ef60 0x55df61b4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2dce03740 4 client.25292 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2c00f0070 con 0x55df6173ef60 2024-04-25T10:55:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.510+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df6173ef60 0x55df61b4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.511+0000 7fc2aeffd640 10 client.25292 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.511+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df6173ef60 0x55df61b4e740 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b25f8d8c7525dd2 server_cookie=93cc83fae0a9fab4 in_seq=0 out_seq=0 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.511+0000 7fc2aeffd640 1 -- 172.21.15.19:0/2377101220 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc2c00f0070 con 0x55df6173ef60 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.511+0000 7fc2aeffd640 10 client.25292 handle_command_reply: tid=0 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 client.25292 shutdown 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 msgr2=0x55df61b30af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b30af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4012730 tx=0x7fc2c4012bf0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 shutdown_connections 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df6173ef60 0x55df61b4e740 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b2b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b30af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2aeffd640 0 client.25292 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:36.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 --2- 172.21.15.19:0/2377101220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b30590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 >> 172.21.15.19:0/2377101220 conn(0x55df617744e0 msgr2=0x55df61b21220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:36.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 shutdown_connections 2024-04-25T10:55:36.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 1 -- 172.21.15.19:0/2377101220 wait complete. 2024-04-25T10:55:36.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.512+0000 7fc2dce03740 20 client.25292 trim_cache size 0 max 16384 2024-04-25T10:55:36.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.549+0000 7fc2dce03740 1 Processor -- start 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.549+0000 7fc2dce03740 1 -- start start 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b2fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b53f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b2fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b2fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51030/0 (socket says 172.21.15.19:51030) 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2ceffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2ceffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b29370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:39952/0 (socket says 172.21.15.19:39952) 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2cf7fe640 1 -- 172.21.15.19:0/297687207 learned_addr learned my addr 172.21.15.19:0/297687207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01ab670 con 0x55df617574d0 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01adef0 con 0x55df6175f6b0 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01abce0 con 0x55df61780390 2024-04-25T10:55:36.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2cffff640 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b53f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2ceffd640 1 -- 172.21.15.19:0/297687207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 msgr2=0x55df61b53f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b53f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.550+0000 7fc2ceffd640 1 -- 172.21.15.19:0/297687207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 msgr2=0x55df61b2fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.551+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b2fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.551+0000 7fc2ceffd640 1 -- 172.21.15.19:0/297687207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df617580f0 con 0x55df6175f6b0 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.551+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b2fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.551+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b29370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4010a40 tx=0x7fc2c4005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12bdb5d57ee32517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.551+0000 7fc2b57fa640 1 -- 172.21.15.19:0/297687207 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4029070 con 0x55df6175f6b0 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.551+0000 7fc2b57fa640 1 -- 172.21.15.19:0/297687207 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c400fdd0 con 0x55df6175f6b0 2024-04-25T10:55:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.551+0000 7fc2b57fa640 1 -- 172.21.15.19:0/297687207 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c401c770 con 0x55df6175f6b0 2024-04-25T10:55:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.552+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 msgr2=0x55df61b29370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.552+0000 7fc2dce03740 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b29370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4010a40 tx=0x7fc2c4005730 comp rx=0 tx=0).stop 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 shutdown_connections 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df61780390 0x55df61b53f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df6175f6b0 0x55df61b29370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 --2- 172.21.15.19:0/297687207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df617574d0 0x55df61b2fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 >> 172.21.15.19:0/297687207 conn(0x55df617744e0 msgr2=0x55df61b26070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 shutdown_connections 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 -- 172.21.15.19:0/297687207 wait complete. 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.553+0000 7fc2dce03740 1 -- start start 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 1 Processor -- start 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df617574d0 0x55df61baec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df6175f6b0 0x55df61baf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b5cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01ac3b0 con 0x55df6175f6b0 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01ad820 con 0x55df61780390 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01aca80 con 0x55df617574d0 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2dce03740 10 client.0 _ll_register_callbacks cb 0x7fc2c4009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2ceffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df6175f6b0 0x55df61baf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2ceffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df6175f6b0 0x55df61baf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51034/0 (socket says 172.21.15.19:51034) 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2ceffd640 1 -- 172.21.15.19:0/3108190336 learned_addr learned my addr 172.21.15.19:0/3108190336 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.554+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df617574d0 0x55df61baec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2cffff640 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b5cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2ceffd640 1 -- 172.21.15.19:0/3108190336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df617574d0 msgr2=0x55df61baec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df617574d0 0x55df61baec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2ceffd640 1 -- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 msgr2=0x55df61b5cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b5cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2ceffd640 1 -- 172.21.15.19:0/3108190336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df61a474b0 con 0x55df6175f6b0 2024-04-25T10:55:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2ceffd640 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df6175f6b0 0x55df61baf140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40108d0 tx=0x7fc2c400ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602ccbe0b9487b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2aeffd640 1 -- 172.21.15.19:0/3108190336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4029070 con 0x55df6175f6b0 2024-04-25T10:55:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2aeffd640 1 -- 172.21.15.19:0/3108190336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc2c4005040 con 0x55df6175f6b0 2024-04-25T10:55:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.555+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c400fdd0 con 0x55df6175f6b0 2024-04-25T10:55:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2aeffd640 1 -- 172.21.15.19:0/3108190336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc2c4005240 con 0x55df6175f6b0 2024-04-25T10:55:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 10 client.15606 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55df61bb6050 con 0x55df6175f6b0 2024-04-25T10:55:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2aeffd640 1 -- 172.21.15.19:0/3108190336 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c400fdd0 con 0x55df6175f6b0 2024-04-25T10:55:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2aeffd640 1 -- 172.21.15.19:0/3108190336 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc2c401c420 con 0x55df6175f6b0 2024-04-25T10:55:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 10 client.15606 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 20 client.15606 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df617143a0 0x55df61bb87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 4 client.15606 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.556+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df61a94990 con 0x55df617143a0 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.557+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df617143a0 0x55df61bb87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.557+0000 7fc2aeffd640 10 client.15606 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.557+0000 7fc2cf7fe640 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df617143a0 0x55df61bb87c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6c053bb3eaa8e8 server_cookie=2ba5d9b2fc5836d0 in_seq=0 out_seq=0 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2aeffd640 1 -- 172.21.15.19:0/3108190336 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df61a94990 con 0x55df617143a0 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2aeffd640 10 client.15606 handle_command_reply: tid=0 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 client.15606 shutdown 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df6175f6b0 msgr2=0x55df61baf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df6175f6b0 0x55df61baf140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40108d0 tx=0x7fc2c400ff80 comp rx=0 tx=0).stop 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 shutdown_connections 2024-04-25T10:55:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df617143a0 0x55df61bb87c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df61780390 0x55df61b5cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2aeffd640 0 client.15606 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df6175f6b0 0x55df61baf140 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 --2- 172.21.15.19:0/3108190336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df617574d0 0x55df61baec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 >> 172.21.15.19:0/3108190336 conn(0x55df617744e0 msgr2=0x55df61b351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 shutdown_connections 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.558+0000 7fc2dce03740 1 -- 172.21.15.19:0/3108190336 wait complete. 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2dce03740 20 client.15606 trim_cache size 0 max 16384 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:55:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:55:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:55:36.563 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:55:36.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc2b80afe70 msgr2=0x7fc2b80b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc2b80afe70 0x7fc2b80b2380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55df61b1db60 tx=0x7fc2c80095a0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 msgr2=0x7fc2d0199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d0199460 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e9f0 tx=0x7fc2c000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 shutdown_connections 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc2b80afe70 0x7fc2b80b2380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2d0108ec0 0x7fc2d019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d0108a20 0x7fc2d01999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.559+0000 7fc2d5f4d640 1 --2- 172.21.15.19:0/2780975944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d0103830 0x7fc2d0199460 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.560+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 >> 172.21.15.19:0/2780975944 conn(0x7fc2d0077880 msgr2=0x7fc2d01094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.560+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 shutdown_connections 2024-04-25T10:55:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.560+0000 7fc2d5f4d640 1 -- 172.21.15.19:0/2780975944 wait complete. 2024-04-25T10:55:36.564 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:36 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:55:36.564 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:36 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:36.564 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:36 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:36.575 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:55:36.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 -- 172.21.15.19:0/3608812604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4104280 msgr2=0x7f7cc4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/3608812604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4104280 0x7f7cc4110650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009be0 tx=0x7f7cbc02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 -- 172.21.15.19:0/3608812604 shutdown_connections 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/3608812604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4104280 0x7f7cc4110650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/3608812604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cc4109fd0 0x7f7cc4103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/3608812604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc4109b30 0x7f7cc41036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 -- 172.21.15.19:0/3608812604 >> 172.21.15.19:0/3608812604 conn(0x7f7cc40ff210 msgr2=0x7f7cc4101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 -- 172.21.15.19:0/3608812604 shutdown_connections 2024-04-25T10:55:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.696+0000 7f7cc8a67640 1 -- 172.21.15.19:0/3608812604 wait complete. 2024-04-25T10:55:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.697+0000 7f7cc8a67640 1 Processor -- start 2024-04-25T10:55:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.697+0000 7f7cc8a67640 1 -- start start 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc8a67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cc4104280 0x7f7cc419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc8a67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4109b30 0x7f7cc419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc8a67640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc4109fd0 0x7f7cc419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc8a67640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4113d20 con 0x7f7cc4104280 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cc4104280 0x7f7cc419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cc4104280 0x7f7cc419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51052/0 (socket says 172.21.15.19:51052) 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc259c640 1 -- 172.21.15.19:0/2405398359 learned_addr learned my addr 172.21.15.19:0/2405398359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc8a67640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4113ba0 con 0x7f7cc4109b30 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4113ea0 con 0x7f7cc4109fd0 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.698+0000 7f7cc2d9d640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc4109fd0 0x7f7cc419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.699+0000 7f7cc2d9d640 1 -- 172.21.15.19:0/2405398359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4109b30 msgr2=0x7f7cc419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.699+0000 7f7cc2d9d640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4109b30 0x7f7cc419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.699+0000 7f7cc2d9d640 1 -- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cc4104280 msgr2=0x7f7cc419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.699+0000 7f7cc2d9d640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cc4104280 0x7f7cc419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.699+0000 7f7cc2d9d640 1 -- 172.21.15.19:0/2405398359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc419f500 con 0x7f7cc4109fd0 2024-04-25T10:55:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.699+0000 7f7cc2d9d640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc4109fd0 0x7f7cc419eec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009b30 tx=0x7f7cbc003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.700+0000 7f7cb37fe640 1 -- 172.21.15.19:0/2405398359 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc041020 con 0x7f7cc4109fd0 2024-04-25T10:55:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.700+0000 7f7cb37fe640 1 -- 172.21.15.19:0/2405398359 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cbc0318f0 con 0x7f7cc4109fd0 2024-04-25T10:55:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.700+0000 7f7cb37fe640 1 -- 172.21.15.19:0/2405398359 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc036040 con 0x7f7cc4109fd0 2024-04-25T10:55:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.700+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc419f790 con 0x7f7cc4109fd0 2024-04-25T10:55:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.700+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc419fb40 con 0x7f7cc4109fd0 2024-04-25T10:55:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.701+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc4104e10 con 0x7f7cc4109fd0 2024-04-25T10:55:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.703+0000 7f7cb37fe640 1 -- 172.21.15.19:0/2405398359 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f7cbc044020 con 0x7f7cc4109fd0 2024-04-25T10:55:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.704+0000 7f7cb37fe640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f7cac0aff50 0x7f7cac0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.704+0000 7f7cc259c640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f7cac0aff50 0x7f7cac0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.704+0000 7f7cb37fe640 1 -- 172.21.15.19:0/2405398359 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f7cbc0ffce0 con 0x7f7cc4109fd0 2024-04-25T10:55:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.705+0000 7f7cc259c640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f7cac0aff50 0x7f7cac0b2460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40058c0 tx=0x7f7cb4005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:36.707+0000 7f7cb37fe640 1 -- 172.21.15.19:0/2405398359 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cbc04b020 con 0x7f7cc4109fd0 2024-04-25T10:55:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.112+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cc406a190 con 0x7f7cc4109fd0 2024-04-25T10:55:37.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.113+0000 7f7cb37fe640 1 -- 172.21.15.19:0/2405398359 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f7cbc0c8560 con 0x7f7cc4109fd0 2024-04-25T10:55:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:55:37.114 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:55:37.114 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:55:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f7cac0aff50 msgr2=0x7f7cac0b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f7cac0aff50 0x7f7cac0b2460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40058c0 tx=0x7f7cb4005f90 comp rx=0 tx=0).stop 2024-04-25T10:55:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc4109fd0 msgr2=0x7f7cc419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc4109fd0 0x7f7cc419eec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009b30 tx=0x7f7cbc003b80 comp rx=0 tx=0).stop 2024-04-25T10:55:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 shutdown_connections 2024-04-25T10:55:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f7cac0aff50 0x7f7cac0b2460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc4109fd0 0x7f7cc419eec0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4109b30 0x7f7cc419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.115+0000 7f7cc8a67640 1 --2- 172.21.15.19:0/2405398359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cc4104280 0x7f7cc419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.116+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 >> 172.21.15.19:0/2405398359 conn(0x7f7cc40ff210 msgr2=0x7f7cc41065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.116+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 shutdown_connections 2024-04-25T10:55:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.116+0000 7f7cc8a67640 1 -- 172.21.15.19:0/2405398359 wait complete. 2024-04-25T10:55:37.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:55:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 -- 172.21.15.19:0/4137917743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 msgr2=0x7f00501025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 --2- 172.21.15.19:0/4137917743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f00501025b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f004c009900 tx=0x7f004c02a410 comp rx=0 tx=0).stop 2024-04-25T10:55:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 -- 172.21.15.19:0/4137917743 shutdown_connections 2024-04-25T10:55:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 --2- 172.21.15.19:0/4137917743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f005010d380 0x7f005010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 --2- 172.21.15.19:0/4137917743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0050102af0 0x7f0050102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 --2- 172.21.15.19:0/4137917743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f00501025b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 -- 172.21.15.19:0/4137917743 >> 172.21.15.19:0/4137917743 conn(0x7f0050078100 msgr2=0x7f0050078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 -- 172.21.15.19:0/4137917743 shutdown_connections 2024-04-25T10:55:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.260+0000 7f005869a640 1 -- 172.21.15.19:0/4137917743 wait complete. 2024-04-25T10:55:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 Processor -- start 2024-04-25T10:55:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 -- start start 2024-04-25T10:55:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0050102af0 0x7f005010f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f005010d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005010d380 0x7f005010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0050112be0 con 0x7f0050108a20 2024-04-25T10:55:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0050112a60 con 0x7f005010d380 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f005869a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0050112d60 con 0x7f0050102af0 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f0055c35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f005010d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f0055c35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f005010d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51072/0 (socket says 172.21.15.19:51072) 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f0055c35640 1 -- 172.21.15.19:0/176050428 learned_addr learned my addr 172.21.15.19:0/176050428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f0056436640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0050102af0 0x7f005010f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.261+0000 7f0056c37640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005010d380 0x7f005010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0055c35640 1 -- 172.21.15.19:0/176050428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0050102af0 msgr2=0x7f005010f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0055c35640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0050102af0 0x7f005010f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0055c35640 1 -- 172.21.15.19:0/176050428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005010d380 msgr2=0x7f005010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0055c35640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005010d380 0x7f005010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0055c35640 1 -- 172.21.15.19:0/176050428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005010e650 con 0x7f0050108a20 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0056c37640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005010d380 0x7f005010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0056436640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0050102af0 0x7f005010f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f0055c35640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f005010d860 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f004000e9f0 tx=0x7f004000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f00477fe640 1 -- 172.21.15.19:0/176050428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000ccc0 con 0x7f0050108a20 2024-04-25T10:55:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00501ad1b0 con 0x7f0050108a20 2024-04-25T10:55:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f00477fe640 1 -- 172.21.15.19:0/176050428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00400035f0 con 0x7f0050108a20 2024-04-25T10:55:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f00477fe640 1 -- 172.21.15.19:0/176050428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000ce60 con 0x7f0050108a20 2024-04-25T10:55:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.262+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501ad700 con 0x7f0050108a20 2024-04-25T10:55:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.265+0000 7f00477fe640 1 -- 172.21.15.19:0/176050428 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f00400037a0 con 0x7f0050108a20 2024-04-25T10:55:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.266+0000 7f005f620740 1 Processor -- start 2024-04-25T10:55:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.266+0000 7f00477fe640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f003c0afe80 0x7f003c0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.266+0000 7f00477fe640 1 -- 172.21.15.19:0/176050428 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f00400df7b0 con 0x7f0050108a20 2024-04-25T10:55:37.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.266+0000 7f0056c37640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f003c0afe80 0x7f003c0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.267+0000 7f005f620740 1 -- start start 2024-04-25T10:55:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.267+0000 7f0056c37640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f003c0afe80 0x7f003c0b2390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00480092f0 tx=0x7f0048009280 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.267+0000 7f005f620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d910810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f005f620740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d910d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f005f620740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f0056436640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d910810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f0056436640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d910810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:40028/0 (socket says 172.21.15.19:40028) 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f0056436640 1 -- 172.21.15.19:0/3706307880 learned_addr learned my addr 172.21.15.19:0/3706307880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb8d875d80 con 0x55bb8d54e6b0 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f0055c35640 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d910d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f0056c37640 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb8d82fa20 con 0x55bb8d56f390 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.268+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bb8d871d40 con 0x55bb8d5464d0 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0056436640 1 -- 172.21.15.19:0/3706307880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 msgr2=0x55bb8d910d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0056436640 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d910d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0056436640 1 -- 172.21.15.19:0/3706307880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 msgr2=0x55bb8d91d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0056436640 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0056436640 1 -- 172.21.15.19:0/3706307880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb8d83ac30 con 0x55bb8d56f390 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0055c35640 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d910d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0056c37640 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.269+0000 7f0056436640 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d910810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f004c031440 tx=0x7f004c005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68fa386863684f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.270+0000 7f0044bfe640 1 -- 172.21.15.19:0/3706307880 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c03f070 con 0x55bb8d56f390 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.270+0000 7f0044bfe640 1 -- 172.21.15.19:0/3706307880 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004c004030 con 0x55bb8d56f390 2024-04-25T10:55:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.270+0000 7f0044bfe640 1 -- 172.21.15.19:0/3706307880 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c035070 con 0x55bb8d56f390 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 msgr2=0x55bb8d910810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d910810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f004c031440 tx=0x7f004c005f00 comp rx=0 tx=0).stop 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 shutdown_connections 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d910d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 --2- 172.21.15.19:0/3706307880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d910810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 >> 172.21.15.19:0/3706307880 conn(0x55bb8d5634e0 msgr2=0x55bb8d90cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 shutdown_connections 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 -- 172.21.15.19:0/3706307880 wait complete. 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.271+0000 7f005f620740 1 -- start start 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 1 Processor -- start 2024-04-25T10:55:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d91b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d9271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb8d91dd30 con 0x55bb8d54e6b0 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb8d7fcdc0 con 0x55bb8d56f390 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bb8d91deb0 con 0x55bb8d5464d0 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f005f620740 10 client.0 _ll_register_callbacks cb 0x55bb8d818600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f0055c35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f0055c35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51094/0 (socket says 172.21.15.19:51094) 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f0055c35640 1 -- 172.21.15.19:0/3338423495 learned_addr learned my addr 172.21.15.19:0/3338423495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0056c37640 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d9271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.272+0000 7f0056436640 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d91b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0055c35640 1 -- 172.21.15.19:0/3338423495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 msgr2=0x55bb8d91b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0055c35640 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d91b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0055c35640 1 -- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 msgr2=0x55bb8d9271b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0055c35640 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d9271b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0055c35640 1 -- 172.21.15.19:0/3338423495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb8d5873f0 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0055c35640 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91b780 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00400a4940 tx=0x7f00400a6e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634d668b4fda7348 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0039ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0039ffb640 1 -- 172.21.15.19:0/3338423495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00400c1020 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f0039ffb640 1 -- 172.21.15.19:0/3338423495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f00400a3cc0 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.273+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c004030 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f0039ffb640 1 -- 172.21.15.19:0/3338423495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f00400a1e60 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 10 client.15624 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bb8d6517d0 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f0039ffb640 1 -- 172.21.15.19:0/3338423495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004c004030 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f0039ffb640 1 -- 172.21.15.19:0/3338423495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f00400bcb20 con 0x55bb8d54e6b0 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 10 client.15624 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 20 client.15624 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d52df60 0x55bb8d92d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 4 client.15624 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.274+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb8d883990 con 0x55bb8d52df60 2024-04-25T10:55:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.275+0000 7f0056436640 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d52df60 0x55bb8d92d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.275+0000 7f0039ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.275+0000 7f0056436640 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d52df60 0x55bb8d92d6d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7b78fe2a9cfd00 server_cookie=ee817ee235601a1c in_seq=0 out_seq=0 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f0039ffb640 1 -- 172.21.15.19:0/3338423495 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb8d883990 con 0x55bb8d52df60 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f0039ffb640 10 client.15624 handle_command_reply: tid=0 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 client.15624 shutdown 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 msgr2=0x55bb8d91b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91b780 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00400a4940 tx=0x7f00400a6e40 comp rx=0 tx=0).stop 2024-04-25T10:55:37.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 shutdown_connections 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d52df60 0x55bb8d92d6d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d56f390 0x55bb8d9271b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d54e6b0 0x55bb8d91b780 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 --2- 172.21.15.19:0/3338423495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5464d0 0x55bb8d91b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 >> 172.21.15.19:0/3338423495 conn(0x55bb8d5634e0 msgr2=0x55bb8d912530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f0039ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.276+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 shutdown_connections 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.277+0000 7f005f620740 1 -- 172.21.15.19:0/3338423495 wait complete. 2024-04-25T10:55:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.277+0000 7f005f620740 20 client.15624 trim_cache size 0 max 16384 2024-04-25T10:55:37.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 Processor -- start 2024-04-25T10:55:37.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 -- start start 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d5464d0 0x55bb8d91ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d56f390 0x55bb8d926e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d5033a0 0x55bb8d932e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00501b3d80 con 0x55bb8d5464d0 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00501b6600 con 0x55bb8d5033a0 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f005f620740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00501b43f0 con 0x55bb8d56f390 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f0056436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d5464d0 0x55bb8d91ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f0056436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d5464d0 0x55bb8d91ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51108/0 (socket says 172.21.15.19:51108) 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f0056436640 1 -- 172.21.15.19:0/62946970 learned_addr learned my addr 172.21.15.19:0/62946970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.291+0000 7f0055c35640 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d56f390 0x55bb8d926e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0056c37640 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d5033a0 0x55bb8d932e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0056436640 1 -- 172.21.15.19:0/62946970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d56f390 msgr2=0x55bb8d926e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0056436640 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d56f390 0x55bb8d926e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0056436640 1 -- 172.21.15.19:0/62946970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d5033a0 msgr2=0x55bb8d932e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0056436640 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d5033a0 0x55bb8d932e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0056436640 1 -- 172.21.15.19:0/62946970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb8d916bf0 con 0x55bb8d5464d0 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0055c35640 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d56f390 0x55bb8d926e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f0056436640 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d5464d0 0x55bb8d91ac80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f004c031440 tx=0x7f004c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c90fc791357157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f003bfff640 1 -- 172.21.15.19:0/62946970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c035030 con 0x55bb8d5464d0 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f003bfff640 1 -- 172.21.15.19:0/62946970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004c0054f0 con 0x55bb8d5464d0 2024-04-25T10:55:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.292+0000 7f003bfff640 1 -- 172.21.15.19:0/62946970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c03f040 con 0x55bb8d5464d0 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.293+0000 7f005f620740 1 -- 172.21.15.19:0/62946970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d5464d0 msgr2=0x55bb8d91ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.293+0000 7f005f620740 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d5464d0 0x55bb8d91ac80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f004c031440 tx=0x7f004c02a410 comp rx=0 tx=0).stop 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.293+0000 7f005f620740 1 -- 172.21.15.19:0/62946970 shutdown_connections 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.293+0000 7f005f620740 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d5033a0 0x55bb8d932e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.293+0000 7f005f620740 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d56f390 0x55bb8d926e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.293+0000 7f005f620740 1 --2- 172.21.15.19:0/62946970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d5464d0 0x55bb8d91ac80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.294+0000 7f005f620740 1 -- 172.21.15.19:0/62946970 >> 172.21.15.19:0/62946970 conn(0x55bb8d5634e0 msgr2=0x55bb8d91a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.294+0000 7f005f620740 1 -- 172.21.15.19:0/62946970 shutdown_connections 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.294+0000 7f005f620740 1 -- 172.21.15.19:0/62946970 wait complete. 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.294+0000 7f005f620740 1 -- start start 2024-04-25T10:55:37.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.294+0000 7f005f620740 1 Processor -- start 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f005f620740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5033a0 0x55bb8d91ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f005f620740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d56f390 0x55bb8d91f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f005f620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d9239e0 0x55bb8d994a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f005f620740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00501b4ac0 con 0x55bb8d56f390 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f005f620740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00501b5f30 con 0x55bb8d9239e0 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f005f620740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00501b5190 con 0x55bb8d5033a0 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0055c35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d56f390 0x55bb8d91f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0055c35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d56f390 0x55bb8d91f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51120/0 (socket says 172.21.15.19:51120) 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0055c35640 1 -- 172.21.15.19:0/1653614988 learned_addr learned my addr 172.21.15.19:0/1653614988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0056c37640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d9239e0 0x55bb8d994a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0056436640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5033a0 0x55bb8d91ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f005f620740 10 client.0 _ll_register_callbacks cb 0x7f00400a7090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0056c37640 1 -- 172.21.15.19:0/1653614988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5033a0 msgr2=0x55bb8d91ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0056c37640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5033a0 0x55bb8d91ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0056c37640 1 -- 172.21.15.19:0/1653614988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d56f390 msgr2=0x55bb8d91f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0056c37640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d56f390 0x55bb8d91f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.295+0000 7f0056c37640 1 -- 172.21.15.19:0/1653614988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb8d91f9f0 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f0055c35640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d56f390 0x55bb8d91f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f0056436640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5033a0 0x55bb8d91ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f0056c37640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d9239e0 0x55bb8d994a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55bb8d9106f0 tx=0x7f0048004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ccd5ed66ae6934b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f0039ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f0039ffb640 1 -- 172.21.15.19:0/1653614988 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004802f060 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f0039ffb640 1 -- 172.21.15.19:0/1653614988 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f0048021b20 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c0054f0 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f0039ffb640 1 -- 172.21.15.19:0/1653614988 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0048021d20 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f005f620740 10 client.25319 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.296+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bb8d995f70 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f0039ffb640 1 -- 172.21.15.19:0/1653614988 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004c0054f0 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f0039ffb640 1 -- 172.21.15.19:0/1653614988 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f004800f040 con 0x55bb8d9239e0 2024-04-25T10:55:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f005f620740 10 client.25319 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:37.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f005f620740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:37.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f005f620740 20 client.25319 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:37.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f005f620740 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d9966a0 0x55bb8d998ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:37.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f005f620740 4 client.25319 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:55:37.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb8d87d800 con 0x55bb8d9966a0 2024-04-25T10:55:37.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.297+0000 7f0056436640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d9966a0 0x55bb8d998ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:37.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f0039ffb640 10 client.25319 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:37.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f0056436640 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d9966a0 0x55bb8d998ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a4fee05e22ca5 server_cookie=28bbfa9ef7967c0b in_seq=0 out_seq=0 2024-04-25T10:55:37.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f0039ffb640 1 -- 172.21.15.19:0/1653614988 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb8d87d800 con 0x55bb8d9966a0 2024-04-25T10:55:37.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f0039ffb640 10 client.25319 handle_command_reply: tid=0 2024-04-25T10:55:37.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 client.25319 shutdown 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d9239e0 msgr2=0x55bb8d994a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d9239e0 0x55bb8d994a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55bb8d9106f0 tx=0x7f0048004a40 comp rx=0 tx=0).stop 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 shutdown_connections 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55bb8d9966a0 0x55bb8d998ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb8d9239e0 0x55bb8d994a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f0039ffb640 0 client.25319 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb8d56f390 0x55bb8d91f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 --2- 172.21.15.19:0/1653614988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb8d5033a0 0x55bb8d91ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.298+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 >> 172.21.15.19:0/1653614988 conn(0x55bb8d5634e0 msgr2=0x55bb8d90c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 shutdown_connections 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005f620740 1 -- 172.21.15.19:0/1653614988 wait complete. 2024-04-25T10:55:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005f620740 20 client.25319 trim_cache size 0 max 16384 2024-04-25T10:55:37.304 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:55:37.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f003c0afe80 msgr2=0x7f003c0b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f003c0afe80 0x7f003c0b2390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00480092f0 tx=0x7f0048009280 comp rx=0 tx=0).stop 2024-04-25T10:55:37.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 msgr2=0x7f005010d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f005010d860 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f004000e9f0 tx=0x7f004000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 shutdown_connections 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f003c0afe80 0x7f003c0b2390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005010d380 0x7f005010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0050108a20 0x7f005010d860 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 --2- 172.21.15.19:0/176050428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0050102af0 0x7f005010f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.299+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 >> 172.21.15.19:0/176050428 conn(0x7f0050078100 msgr2=0x7f00500ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.300+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 shutdown_connections 2024-04-25T10:55:37.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:37.300+0000 7f005869a640 1 -- 172.21.15.19:0/176050428 wait complete. 2024-04-25T10:55:37.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:37 smithi019 ceph-mon[29209]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:37 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:55:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:37 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:55:37.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:37 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:37 smithi169 ceph-mon[33049]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:37 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:55:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:37 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:55:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:37 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:37.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:37 smithi112 ceph-mon[32799]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:37 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:55:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:37 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:55:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:37 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:55:38.316 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:55:38.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:55:38.316 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:55:38.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:38 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2405398359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 -- 172.21.15.19:0/2962887167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108ec0 msgr2=0x7f75941031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 --2- 172.21.15.19:0/2962887167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108ec0 0x7f75941031c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7580009900 tx=0x7f758002a410 comp rx=0 tx=0).stop 2024-04-25T10:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 -- 172.21.15.19:0/2962887167 shutdown_connections 2024-04-25T10:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 --2- 172.21.15.19:0/2962887167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594103830 0x7f759410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 --2- 172.21.15.19:0/2962887167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108ec0 0x7f75941031c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 --2- 172.21.15.19:0/2962887167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7594108a20 0x7f7594102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 -- 172.21.15.19:0/2962887167 >> 172.21.15.19:0/2962887167 conn(0x7f7594077880 msgr2=0x7f7594077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 -- 172.21.15.19:0/2962887167 shutdown_connections 2024-04-25T10:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.444+0000 7f759af46640 1 -- 172.21.15.19:0/2962887167 wait complete. 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.445+0000 7f759af46640 1 Processor -- start 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.445+0000 7f759af46640 1 -- start start 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.445+0000 7f759af46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594103830 0x7f759419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.445+0000 7f759af46640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108a20 0x7f759419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f759af46640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7594108ec0 0x7f759419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f759af46640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7594112c10 con 0x7f7594108a20 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f759af46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7594112a90 con 0x7f7594103830 2024-04-25T10:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108a20 0x7f759419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f759af46640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7594112d90 con 0x7f7594108ec0 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108a20 0x7f759419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51130/0 (socket says 172.21.15.19:51130) 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 -- 172.21.15.19:0/3062618630 learned_addr learned my addr 172.21.15.19:0/3062618630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7598ce2640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594103830 0x7f759419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f75994e3640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7594108ec0 0x7f759419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 -- 172.21.15.19:0/3062618630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7594108ec0 msgr2=0x7f759419c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7594108ec0 0x7f759419c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 -- 172.21.15.19:0/3062618630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594103830 msgr2=0x7f759419b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594103830 0x7f759419b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 -- 172.21.15.19:0/3062618630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75941a6920 con 0x7f7594108a20 2024-04-25T10:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.446+0000 7f7593fff640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108a20 0x7f759419bb50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7580006010 tx=0x7f7580004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.447+0000 7f7591ffb640 1 -- 172.21.15.19:0/3062618630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758003f030 con 0x7f7594108a20 2024-04-25T10:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.447+0000 7f7591ffb640 1 -- 172.21.15.19:0/3062618630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7580003aa0 con 0x7f7594108a20 2024-04-25T10:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.447+0000 7f7591ffb640 1 -- 172.21.15.19:0/3062618630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580035070 con 0x7f7594108a20 2024-04-25T10:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.447+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75941a6bb0 con 0x7f7594108a20 2024-04-25T10:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.447+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a6f90 con 0x7f7594108a20 2024-04-25T10:55:38.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 Processor -- start 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 -- start start 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f7591ffb640 1 -- 172.21.15.19:0/3062618630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f7580042030 con 0x7f7594108a20 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 0x55c934cdb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934cdbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cdc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c934c41b00 con 0x55c934912550 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c934bdd920 con 0x55c93491a730 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75a1df9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c934c3dac0 con 0x55c93493b410 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f7598ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934cdbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f7598ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934cdbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51132/0 (socket says 172.21.15.19:51132) 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f7598ce2640 1 -- 172.21.15.19:0/1822425749 learned_addr learned my addr 172.21.15.19:0/1822425749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.451+0000 7f75994e3640 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 0x55c934cdb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7593fff640 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cdc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7598ce2640 1 -- 172.21.15.19:0/1822425749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 msgr2=0x55c934cdb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7598ce2640 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 0x55c934cdb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7598ce2640 1 -- 172.21.15.19:0/1822425749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 msgr2=0x55c934cdc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7598ce2640 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cdc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7598ce2640 1 -- 172.21.15.19:0/1822425749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c934a1b7e0 con 0x55c934912550 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7593fff640 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cdc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7591ffb640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f75840afd80 0x7f75840b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7598ce2640 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934cdbb60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7588009ca0 tx=0x7f7588011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f240d4bd33b13bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.452+0000 7f7591ffb640 1 -- 172.21.15.19:0/3062618630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f75800ff6d0 con 0x7f7594108a20 2024-04-25T10:55:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.453+0000 7f757effd640 1 -- 172.21.15.19:0/1822425749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758801e070 con 0x55c934912550 2024-04-25T10:55:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.453+0000 7f75994e3640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f75840afd80 0x7f75840b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.453+0000 7f757effd640 1 -- 172.21.15.19:0/1822425749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7588019ae0 con 0x55c934912550 2024-04-25T10:55:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.453+0000 7f757effd640 1 -- 172.21.15.19:0/1822425749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758800ea20 con 0x55c934912550 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 -- 172.21.15.19:0/1822425749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 msgr2=0x55c934cdbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934cdbb60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7588009ca0 tx=0x7f7588011810 comp rx=0 tx=0).stop 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75994e3640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f75840afd80 0x7f75840b2290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55c934cd8790 tx=0x7f758c009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 -- 172.21.15.19:0/1822425749 shutdown_connections 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cdc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934cdbb60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1822425749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 0x55c934cdb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 -- 172.21.15.19:0/1822425749 >> 172.21.15.19:0/1822425749 conn(0x55c93492f560 msgr2=0x55c934cd7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 -- 172.21.15.19:0/1822425749 shutdown_connections 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 -- 172.21.15.19:0/1822425749 wait complete. 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 -- start start 2024-04-25T10:55:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.454+0000 7f75a1df9740 1 Processor -- start 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934ce5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934ce6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 0x55c934cf1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f75a1df9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c934ce8970 con 0x55c934912550 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f75a1df9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c934bdb5d0 con 0x55c93491a730 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f75a1df9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c934ce8af0 con 0x55c93493b410 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f75a1df9740 10 client.0 _ll_register_callbacks cb 0x55c934bc9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f7598ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934ce5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f7598ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934ce5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51146/0 (socket says 172.21.15.19:51146) 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f7598ce2640 1 -- 172.21.15.19:0/1489530161 learned_addr learned my addr 172.21.15.19:0/1489530161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.455+0000 7f7593fff640 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934ce6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7593fff640 1 -- 172.21.15.19:0/1489530161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 msgr2=0x55c934cf1e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7593fff640 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 0x55c934cf1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7593fff640 1 -- 172.21.15.19:0/1489530161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 msgr2=0x55c934ce5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7593fff640 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934ce5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7593fff640 1 -- 172.21.15.19:0/1489530161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c934a7bf70 con 0x55c93491a730 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7598ce2640 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934ce5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7593fff640 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934ce6150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f75800fccd0 tx=0x7f758002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0294e4f6f0e796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7573fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7573fff640 1 -- 172.21.15.19:0/1489530161 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580049070 con 0x55c93491a730 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7573fff640 1 -- 172.21.15.19:0/1489530161 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f75800c7980 con 0x55c93491a730 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7588019ae0 con 0x55c93491a730 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f7573fff640 1 -- 172.21.15.19:0/1489530161 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f75800c74f0 con 0x55c93491a730 2024-04-25T10:55:38.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f75a1df9740 10 client.25331 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.456+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c934913170 con 0x55c93491a730 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f7573fff640 1 -- 172.21.15.19:0/1489530161 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7588019ae0 con 0x55c93491a730 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f7573fff640 1 -- 172.21.15.19:0/1489530161 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f75800d00f0 con 0x55c93491a730 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f75a1df9740 10 client.25331 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f75a1df9740 10 client.25331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f75a1df9740 20 client.25331 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9348fa1f0 0x55c934cf8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f75a1df9740 4 client.25331 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c934c4f710 con 0x55c9348fa1f0 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.457+0000 7f7598ce2640 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9348fa1f0 0x55c934cf8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.458+0000 7f7573fff640 10 client.25331 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.458+0000 7f7598ce2640 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9348fa1f0 0x55c934cf8030 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abcc6fd5467a80b server_cookie=1df154c026216183 in_seq=0 out_seq=0 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.458+0000 7f7573fff640 1 -- 172.21.15.19:0/1489530161 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c934c4f710 con 0x55c9348fa1f0 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.458+0000 7f7573fff640 10 client.25331 handle_command_reply: tid=0 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.458+0000 7f75a1df9740 1 client.25331 shutdown 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 msgr2=0x55c934ce6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934ce6150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f75800fccd0 tx=0x7f758002a410 comp rx=0 tx=0).stop 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 shutdown_connections 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9348fa1f0 0x55c934cf8030 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c93493b410 0x55c934cf1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f7573fff640 0 client.25331 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934ce6150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 --2- 172.21.15.19:0/1489530161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c934912550 0x55c934ce5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 >> 172.21.15.19:0/1489530161 conn(0x55c93492f560 msgr2=0x55c934cdd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 shutdown_connections 2024-04-25T10:55:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 1 -- 172.21.15.19:0/1489530161 wait complete. 2024-04-25T10:55:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.459+0000 7f75a1df9740 20 client.25331 trim_cache size 0 max 16384 2024-04-25T10:55:38.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.496+0000 7f75a1df9740 1 Processor -- start 2024-04-25T10:55:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.496+0000 7f75a1df9740 1 -- start start 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9348fa1f0 0x55c934cd7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934cfd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c93491a730 0x55c934cfdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75a1df9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ad8c0 con 0x55c93491a730 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75a1df9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75941b0140 con 0x55c9348fa1f0 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75a1df9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75941adf30 con 0x55c934912550 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c93491a730 0x55c934cfdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c93491a730 0x55c934cfdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51156/0 (socket says 172.21.15.19:51156) 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 -- 172.21.15.19:0/4197919827 learned_addr learned my addr 172.21.15.19:0/4197919827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f7593fff640 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934cfd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f7598ce2640 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9348fa1f0 0x55c934cd7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 -- 172.21.15.19:0/4197919827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 msgr2=0x55c934cfd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934cfd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 -- 172.21.15.19:0/4197919827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9348fa1f0 msgr2=0x55c934cd7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9348fa1f0 0x55c934cd7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f75994e3640 1 -- 172.21.15.19:0/4197919827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c934ce8130 con 0x55c93491a730 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.497+0000 7f7598ce2640 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9348fa1f0 0x55c934cd7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.498+0000 7f75994e3640 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c93491a730 0x55c934cfdd30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f758c009fd0 tx=0x7f758c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb29a9e1ac24b08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.498+0000 7f757e7fc640 1 -- 172.21.15.19:0/4197919827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c00f790 con 0x55c93491a730 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.498+0000 7f757e7fc640 1 -- 172.21.15.19:0/4197919827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f758c00fde0 con 0x55c93491a730 2024-04-25T10:55:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.498+0000 7f757e7fc640 1 -- 172.21.15.19:0/4197919827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c027400 con 0x55c93491a730 2024-04-25T10:55:38.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:38 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2405398359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:55:38.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.499+0000 7f75a1df9740 1 -- 172.21.15.19:0/4197919827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c93491a730 msgr2=0x55c934cfdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.499+0000 7f75a1df9740 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c93491a730 0x55c934cfdd30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f758c009fd0 tx=0x7f758c014000 comp rx=0 tx=0).stop 2024-04-25T10:55:38.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.499+0000 7f75a1df9740 1 -- 172.21.15.19:0/4197919827 shutdown_connections 2024-04-25T10:55:38.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.499+0000 7f75a1df9740 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c93491a730 0x55c934cfdd30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.499+0000 7f75a1df9740 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934cfd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.500+0000 7f75a1df9740 1 --2- 172.21.15.19:0/4197919827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9348fa1f0 0x55c934cd7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.500+0000 7f75a1df9740 1 -- 172.21.15.19:0/4197919827 >> 172.21.15.19:0/4197919827 conn(0x55c93492f560 msgr2=0x55c934cd7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.500+0000 7f75a1df9740 1 -- 172.21.15.19:0/4197919827 shutdown_connections 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.500+0000 7f75a1df9740 1 -- 172.21.15.19:0/4197919827 wait complete. 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.500+0000 7f75a1df9740 1 -- start start 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.500+0000 7f75a1df9740 1 Processor -- start 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9348fa1f0 0x55c934d5b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934d5bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75a1df9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cedec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75a1df9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ae600 con 0x55c9348fa1f0 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f7598ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9348fa1f0 0x55c934d5b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75a1df9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75941afa70 con 0x55c93491a730 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f7598ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9348fa1f0 0x55c934d5b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51166/0 (socket says 172.21.15.19:51166) 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f7598ce2640 1 -- 172.21.15.19:0/3021148422 learned_addr learned my addr 172.21.15.19:0/3021148422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75a1df9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75941aecd0 con 0x55c934912550 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75994e3640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cedec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f75a1df9740 10 client.0 _ll_register_callbacks cb 0x7f7580003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.501+0000 7f7593fff640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934d5bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75994e3640 1 -- 172.21.15.19:0/3021148422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 msgr2=0x55c934d5bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75994e3640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934d5bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75994e3640 1 -- 172.21.15.19:0/3021148422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9348fa1f0 msgr2=0x55c934d5b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75994e3640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9348fa1f0 0x55c934d5b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75994e3640 1 -- 172.21.15.19:0/3021148422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c934cee630 con 0x55c93491a730 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f7598ce2640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9348fa1f0 0x55c934d5b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75994e3640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cedec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d30 tx=0x7f758c01fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c945a923d3b36db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f7573fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f7573fff640 1 -- 172.21.15.19:0/3021148422 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c004c50 con 0x55c93491a730 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f7573fff640 1 -- 172.21.15.19:0/3021148422 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f758c004df0 con 0x55c93491a730 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758c00fde0 con 0x55c93491a730 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f7573fff640 1 -- 172.21.15.19:0/3021148422 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f758c00f590 con 0x55c93491a730 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.502+0000 7f75a1df9740 10 client.25337 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c934d647b0 con 0x55c93491a730 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f7573fff640 1 -- 172.21.15.19:0/3021148422 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f758c00fde0 con 0x55c93491a730 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f7573fff640 1 -- 172.21.15.19:0/3021148422 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f758c0052b0 con 0x55c93491a730 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f75a1df9740 10 client.25337 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f75a1df9740 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f75a1df9740 20 client.25337 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f75a1df9740 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c93493b410 0x55c934d66d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f75a1df9740 4 client.25337 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.503+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c934c49580 con 0x55c93493b410 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.504+0000 7f7598ce2640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c93493b410 0x55c934d66d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.504+0000 7f7573fff640 10 client.25337 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.504+0000 7f7598ce2640 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c93493b410 0x55c934d66d10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2cbeeb40a316b2 server_cookie=13197a9f8f1d65d9 in_seq=0 out_seq=0 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.505+0000 7f7573fff640 1 -- 172.21.15.19:0/3021148422 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c934c49580 con 0x55c93493b410 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.505+0000 7f7573fff640 10 client.25337 handle_command_reply: tid=0 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.505+0000 7f75a1df9740 1 client.25337 shutdown 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 msgr2=0x55c934cedec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cedec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d30 tx=0x7f758c01fdb0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 shutdown_connections 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c93493b410 0x55c934d66d10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c93491a730 0x55c934cedec0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c934912550 0x55c934d5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f7573fff640 0 client.25337 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 --2- 172.21.15.19:0/3021148422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9348fa1f0 0x55c934d5b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 >> 172.21.15.19:0/3021148422 conn(0x55c93492f560 msgr2=0x55c934ce42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 shutdown_connections 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 1 -- 172.21.15.19:0/3021148422 wait complete. 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.506+0000 7f75a1df9740 20 client.25337 trim_cache size 0 max 16384 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "39161545-ca78-4909-8c59-57fafd93d647", 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:55:38.512 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:55:38.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f75840afd80 msgr2=0x7f75840b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f75840afd80 0x7f75840b2290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55c934cd8790 tx=0x7f758c009600 comp rx=0 tx=0).stop 2024-04-25T10:55:38.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108a20 msgr2=0x7f759419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108a20 0x7f759419bb50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7580006010 tx=0x7f7580004060 comp rx=0 tx=0).stop 2024-04-25T10:55:38.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 shutdown_connections 2024-04-25T10:55:38.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f75840afd80 0x7f75840b2290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7594108ec0 0x7f759419c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7594108a20 0x7f759419bb50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 --2- 172.21.15.19:0/3062618630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594103830 0x7f759419b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 >> 172.21.15.19:0/3062618630 conn(0x7f7594077880 msgr2=0x7f7594104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 shutdown_connections 2024-04-25T10:55:38.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.507+0000 7f759af46640 1 -- 172.21.15.19:0/3062618630 wait complete. 2024-04-25T10:55:38.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:55:38.524 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:55:38.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:38 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2405398359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:55:38.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 -- 172.21.15.19:0/1257355212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 msgr2=0x7f82c410f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/1257355212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 0x7f82c410f280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009900 tx=0x7f82bc02c620 comp rx=0 tx=0).stop 2024-04-25T10:55:38.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 -- 172.21.15.19:0/1257355212 shutdown_connections 2024-04-25T10:55:38.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/1257355212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 0x7f82c410f280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/1257355212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4107020 0x7f82c4107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/1257355212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 0x7f82c4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 -- 172.21.15.19:0/1257355212 >> 172.21.15.19:0/1257355212 conn(0x7f82c40fdf10 msgr2=0x7f82c4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 -- 172.21.15.19:0/1257355212 shutdown_connections 2024-04-25T10:55:38.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.738+0000 7f82c9b9d640 1 -- 172.21.15.19:0/1257355212 wait complete. 2024-04-25T10:55:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.739+0000 7f82c9b9d640 1 Processor -- start 2024-04-25T10:55:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.739+0000 7f82c9b9d640 1 -- start start 2024-04-25T10:55:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c9b9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 0x7f82c419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c9b9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4107020 0x7f82c419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c9b9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 0x7f82c41a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c9b9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41118f0 con 0x7f82c4102a90 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 0x7f82c419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c9b9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111770 con 0x7f82c4107ae0 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c9b9d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111a70 con 0x7f82c4107020 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 0x7f82c419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51188/0 (socket says 172.21.15.19:51188) 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 -- 172.21.15.19:0/3887049226 learned_addr learned my addr 172.21.15.19:0/3887049226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c939c640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 0x7f82c41a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4107020 0x7f82c419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 -- 172.21.15.19:0/3887049226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4107020 msgr2=0x7f82c419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4107020 0x7f82c419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 -- 172.21.15.19:0/3887049226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 msgr2=0x7f82c41a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 0x7f82c41a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.740+0000 7f82c8b9b640 1 -- 172.21.15.19:0/3887049226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41a15a0 con 0x7f82c4102a90 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.741+0000 7f82c8b9b640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 0x7f82c419c740 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009d50 tx=0x7f82b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.741+0000 7f82c1ffb640 1 -- 172.21.15.19:0/3887049226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8003b70 con 0x7f82c4102a90 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.741+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a1890 con 0x7f82c4102a90 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.741+0000 7f82c1ffb640 1 -- 172.21.15.19:0/3887049226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b8003d10 con 0x7f82c4102a90 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.742+0000 7f82c1ffb640 1 -- 172.21.15.19:0/3887049226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b80188d0 con 0x7f82c4102a90 2024-04-25T10:55:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.742+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a9210 con 0x7f82c4102a90 2024-04-25T10:55:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.745+0000 7f82d1d86740 1 Processor -- start 2024-04-25T10:55:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.745+0000 7f82c1ffb640 1 -- 172.21.15.19:0/3887049226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f82b8004560 con 0x7f82c4102a90 2024-04-25T10:55:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.745+0000 7f82d1d86740 1 -- start start 2024-04-25T10:55:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2bc9390 0x559fd2f69c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f76ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82d1d86740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fd2ecfd80 con 0x559fd2ba86b0 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82d1d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fd2e898b0 con 0x559fd2bc9390 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82d1d86740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559fd2ecbd40 con 0x559fd2ba04d0 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f76ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f76ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51200/0 (socket says 172.21.15.19:51200) 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c939c640 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2bc9390 0x559fd2f69c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c3fff640 1 -- 172.21.15.19:0/3329138827 learned_addr learned my addr 172.21.15.19:0/3329138827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c8b9b640 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c939c640 1 -- 172.21.15.19:0/3329138827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 msgr2=0x559fd2f6a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c939c640 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f6a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c939c640 1 -- 172.21.15.19:0/3329138827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 msgr2=0x559fd2f76ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c939c640 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f76ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.746+0000 7f82c939c640 1 -- 172.21.15.19:0/3329138827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd2e5aa40 con 0x559fd2bc9390 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.747+0000 7f82c939c640 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2bc9390 0x559fd2f69c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006010 tx=0x7f82bc00e410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aef5a6a2476bab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.747+0000 7f82c1ffb640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f82b40afdf0 0x7f82b40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.747+0000 7f82aeffd640 1 -- 172.21.15.19:0/3329138827 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc037040 con 0x559fd2bc9390 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.747+0000 7f82aeffd640 1 -- 172.21.15.19:0/3329138827 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82bc004030 con 0x559fd2bc9390 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.747+0000 7f82aeffd640 1 -- 172.21.15.19:0/3329138827 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc005320 con 0x559fd2bc9390 2024-04-25T10:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.747+0000 7f82c8b9b640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f82b40afdf0 0x7f82b40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82c1ffb640 1 -- 172.21.15.19:0/3887049226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f82b80dfb40 con 0x7f82c4102a90 2024-04-25T10:55:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 -- 172.21.15.19:0/3329138827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2bc9390 msgr2=0x559fd2f69c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82c8b9b640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f82b40afdf0 0x7f82b40b2300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f82b80dd140 tx=0x7f82b80e34a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2bc9390 0x559fd2f69c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006010 tx=0x7f82bc00e410 comp rx=0 tx=0).stop 2024-04-25T10:55:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 -- 172.21.15.19:0/3329138827 shutdown_connections 2024-04-25T10:55:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f76ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f6a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3329138827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2bc9390 0x559fd2f69c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 -- 172.21.15.19:0/3329138827 >> 172.21.15.19:0/3329138827 conn(0x559fd2bbd4e0 msgr2=0x559fd2f66400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 -- 172.21.15.19:0/3329138827 shutdown_connections 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.748+0000 7f82d1d86740 1 -- 172.21.15.19:0/3329138827 wait complete. 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.749+0000 7f82d1d86740 1 -- start start 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.749+0000 7f82d1d86740 1 Processor -- start 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2f74c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2bc9390 0x559fd2f806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82d1d86740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fd2f77170 con 0x559fd2bc9390 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82d1d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fd2e56d10 con 0x559fd2ba86b0 2024-04-25T10:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82c939c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2bc9390 0x559fd2f806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82c939c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2bc9390 0x559fd2f806b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51202/0 (socket says 172.21.15.19:51202) 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82c939c640 1 -- 172.21.15.19:0/3737725394 learned_addr learned my addr 172.21.15.19:0/3737725394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2f74c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559fd2f772f0 con 0x559fd2ba04d0 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82c939c640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82d1d86740 10 client.0 _ll_register_callbacks cb 0x559fd2e83210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.750+0000 7f82c3fff640 1 -- 172.21.15.19:0/3737725394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 msgr2=0x559fd2f746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82c3fff640 1 -- 172.21.15.19:0/3737725394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2bc9390 msgr2=0x559fd2f806b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2bc9390 0x559fd2f806b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82c3fff640 1 -- 172.21.15.19:0/3737725394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd2cab7d0 con 0x559fd2ba86b0 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82c939c640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2bc9390 0x559fd2f806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2f74c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009fd0 tx=0x7f82b00138f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8064e3543d800186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82a3fff640 10 client.25349 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82bc004030 con 0x559fd2ba86b0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82a3fff640 1 -- 172.21.15.19:0/3737725394 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0004300 con 0x559fd2ba86b0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.751+0000 7f82a3fff640 1 -- 172.21.15.19:0/3737725394 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f82b00035f0 con 0x559fd2ba86b0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82a3fff640 1 -- 172.21.15.19:0/3737725394 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82b00044a0 con 0x559fd2ba86b0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82d1d86740 10 client.25349 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=20}) -- 0x559fd2be13f0 con 0x559fd2ba86b0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82a3fff640 1 -- 172.21.15.19:0/3737725394 <== mon.2 v2:172.21.15.112:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f82b000f040 con 0x559fd2ba86b0 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82d1d86740 10 client.25349 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82d1d86740 10 client.25349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82d1d86740 20 client.25349 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.752+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2b87f60 0x559fd2f86620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.753+0000 7f82d1d86740 4 client.25349 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.753+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f82bc004030 con 0x559fd2b87f60 2024-04-25T10:55:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.753+0000 7f82c939c640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2b87f60 0x559fd2f86620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.753+0000 7f82a3fff640 10 client.25349 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.753+0000 7f82c939c640 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2b87f60 0x559fd2f86620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b900b9a8d94be44 server_cookie=5caf5cf88748413 in_seq=0 out_seq=0 2024-04-25T10:55:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.754+0000 7f82a3fff640 1 -- 172.21.15.19:0/3737725394 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f82bc004030 con 0x559fd2b87f60 2024-04-25T10:55:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.754+0000 7f82a3fff640 10 client.25349 handle_command_reply: tid=0 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.754+0000 7f82d1d86740 1 client.25349 shutdown 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.754+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 msgr2=0x559fd2f74c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.754+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2f74c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009fd0 tx=0x7f82b00138f0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.754+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 shutdown_connections 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.754+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2b87f60 0x559fd2f86620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82a3fff640 0 client.25349 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2bc9390 0x559fd2f806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2f74c30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3737725394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2ba04d0 0x559fd2f746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 >> 172.21.15.19:0/3737725394 conn(0x559fd2bbd4e0 msgr2=0x559fd2f6b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 shutdown_connections 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82d1d86740 1 -- 172.21.15.19:0/3737725394 wait complete. 2024-04-25T10:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.755+0000 7f82d1d86740 20 client.25349 trim_cache size 0 max 16384 2024-04-25T10:55:38.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 Processor -- start 2024-04-25T10:55:38.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 -- start start 2024-04-25T10:55:38.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f8bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba04d0 0x559fd2f8c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f80290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41af8c0 con 0x559fd2ba86b0 2024-04-25T10:55:38.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41b2140 con 0x559fd2ba04d0 2024-04-25T10:55:38.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.791+0000 7f82d1d86740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41aff30 con 0x559fd2b87f60 2024-04-25T10:55:38.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f80290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f80290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51214/0 (socket says 172.21.15.19:51214) 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 -- 172.21.15.19:0/2311327174 learned_addr learned my addr 172.21.15.19:0/2311327174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f8bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c3fff640 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba04d0 0x559fd2f8c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 -- 172.21.15.19:0/2311327174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 msgr2=0x559fd2f8bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f8bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 -- 172.21.15.19:0/2311327174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba04d0 msgr2=0x559fd2f8c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba04d0 0x559fd2f8c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 -- 172.21.15.19:0/2311327174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd2ba10f0 con 0x559fd2ba86b0 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c3fff640 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba04d0 0x559fd2f8c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.792+0000 7f82c939c640 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f80290 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005f70 tx=0x7f82bc048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b69c994eeee97e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.793+0000 7f82ae7fc640 1 -- 172.21.15.19:0/2311327174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc037040 con 0x559fd2ba86b0 2024-04-25T10:55:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.793+0000 7f82ae7fc640 1 -- 172.21.15.19:0/2311327174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82bc05ed00 con 0x559fd2ba86b0 2024-04-25T10:55:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.793+0000 7f82ae7fc640 1 -- 172.21.15.19:0/2311327174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0544d0 con 0x559fd2ba86b0 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 -- 172.21.15.19:0/2311327174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 msgr2=0x559fd2f80290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f80290 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005f70 tx=0x7f82bc048040 comp rx=0 tx=0).stop 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 -- 172.21.15.19:0/2311327174 shutdown_connections 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba86b0 0x559fd2f80290 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba04d0 0x559fd2f8c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 --2- 172.21.15.19:0/2311327174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f8bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 -- 172.21.15.19:0/2311327174 >> 172.21.15.19:0/2311327174 conn(0x559fd2bbd4e0 msgr2=0x559fd2f66350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 -- 172.21.15.19:0/2311327174 shutdown_connections 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 -- 172.21.15.19:0/2311327174 wait complete. 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.794+0000 7f82d1d86740 1 -- start start 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.795+0000 7f82d1d86740 1 Processor -- start 2024-04-25T10:55:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.795+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f77bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.795+0000 7f82d1d86740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba04d0 0x559fd2f78130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.795+0000 7f82c939c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f77bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba04d0 0x559fd2f78130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba04d0 0x559fd2f78130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51220/0 (socket says 172.21.15.19:51220) 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c939c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f77bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:51840/0 (socket says 172.21.15.19:51840) 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 -- 172.21.15.19:0/3595587263 learned_addr learned my addr 172.21.15.19:0/3595587263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2fe6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41b0600 con 0x559fd2ba04d0 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41b1a70 con 0x559fd2ba86b0 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41b0cd0 con 0x559fd2b87f60 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 -- 172.21.15.19:0/3595587263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 msgr2=0x559fd2f77bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f77bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 -- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 msgr2=0x559fd2fe6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c939c640 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2fe6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2fe6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c3fff640 1 -- 172.21.15.19:0/3595587263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd2e904b0 con 0x559fd2ba04d0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.796+0000 7f82c939c640 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2fe6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82d1d86740 10 client.0 _ll_register_callbacks cb 0x7f82b0005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82c3fff640 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba04d0 0x559fd2f78130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009670 tx=0x7f82b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a941142b3a40c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82bc05ed00 con 0x559fd2ba04d0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82a3fff640 1 -- 172.21.15.19:0/3595587263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0033030 con 0x559fd2ba04d0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82a3fff640 1 -- 172.21.15.19:0/3595587263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f82b002b070 con 0x559fd2ba04d0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82a3fff640 1 -- 172.21.15.19:0/3595587263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82b0004300 con 0x559fd2ba04d0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82d1d86740 10 client.15666 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.797+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=20}) -- 0x559fd2fedf00 con 0x559fd2ba04d0 2024-04-25T10:55:38.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82a3fff640 1 -- 172.21.15.19:0/3595587263 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f82b0011ac0 con 0x559fd2ba04d0 2024-04-25T10:55:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82d1d86740 10 client.15666 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82d1d86740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82d1d86740 20 client.15666 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2bc9390 0x559fd2ff0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82d1d86740 4 client.15666 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:55:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f82bc05ed00 con 0x559fd2bc9390 2024-04-25T10:55:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.798+0000 7f82c939c640 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2bc9390 0x559fd2ff0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82a3fff640 10 client.15666 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82c939c640 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2bc9390 0x559fd2ff0600 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47569b6df09387a server_cookie=c242a5a3c209e383 in_seq=0 out_seq=0 2024-04-25T10:55:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82a3fff640 1 -- 172.21.15.19:0/3595587263 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f82bc05ed00 con 0x559fd2bc9390 2024-04-25T10:55:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82a3fff640 10 client.15666 handle_command_reply: tid=0 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 client.15666 shutdown 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba04d0 msgr2=0x559fd2f78130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba04d0 0x559fd2f78130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009670 tx=0x7f82b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 shutdown_connections 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559fd2bc9390 0x559fd2ff0600 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd2ba86b0 0x559fd2fe6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fd2ba04d0 0x559fd2f78130 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 --2- 172.21.15.19:0/3595587263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fd2b87f60 0x559fd2f77bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 >> 172.21.15.19:0/3595587263 conn(0x559fd2bbd4e0 msgr2=0x559fd2f66350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82a3fff640 0 client.15666 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 shutdown_connections 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.799+0000 7f82d1d86740 1 -- 172.21.15.19:0/3595587263 wait complete. 2024-04-25T10:55:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82d1d86740 20 client.15666 trim_cache size 0 max 16384 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: "39161545-ca78-4909-8c59-57fafd93d647": { 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "39161545-ca78-4909-8c59-57fafd93d647", 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:55:38.805 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:55:38.806 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:55:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f82b40afdf0 msgr2=0x7f82b40b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f82b40afdf0 0x7f82b40b2300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f82b80dd140 tx=0x7f82b80e34a0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 msgr2=0x7f82c419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 0x7f82c419c740 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009d50 tx=0x7f82b800c6a0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 shutdown_connections 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f82b40afdf0 0x7f82b40b2300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c4107ae0 0x7f82c41a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4107020 0x7f82c419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 --2- 172.21.15.19:0/3887049226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82c4102a90 0x7f82c419c740 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.800+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 >> 172.21.15.19:0/3887049226 conn(0x7f82c40fdf10 msgr2=0x7f82c4105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.801+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 shutdown_connections 2024-04-25T10:55:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:55:38.801+0000 7f82c9b9d640 1 -- 172.21.15.19:0/3887049226 wait complete. 2024-04-25T10:55:38.816 INFO:tasks.cephfs.filesystem:scrub status for tag:39161545-ca78-4909-8c59-57fafd93d647 - {'path': '/', 'tag': '39161545-ca78-4909-8c59-57fafd93d647', 'options': 'recursive,force'} 2024-04-25T10:55:39.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:39 smithi019 ceph-mon[29209]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:39.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:39 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:55:39.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:39 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:39.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:39 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:55:39.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:39 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:39 smithi169 ceph-mon[33049]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:39 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:55:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:39 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:39 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:55:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:39 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:39 smithi112 ceph-mon[32799]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:39 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:55:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:39 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:39 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:55:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:39 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:55:41.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:41 smithi019 ceph-mon[29209]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:41.361 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:41 smithi112 ceph-mon[32799]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:41.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:41 smithi169 ceph-mon[33049]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 -- 172.21.15.19:0/3349581410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 msgr2=0x7fb218102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/3349581410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb218102b40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009900 tx=0x7fb20c02a410 comp rx=0 tx=0).stop 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 -- 172.21.15.19:0/3349581410 shutdown_connections 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/3349581410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2181031b0 0x7fb21810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/3349581410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb218102b40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/3349581410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb218108a50 0x7fb218102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 -- 172.21.15.19:0/3349581410 >> 172.21.15.19:0/3349581410 conn(0x7fb2180fe260 msgr2=0x7fb218100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 -- 172.21.15.19:0/3349581410 shutdown_connections 2024-04-25T10:55:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.221+0000 7fb21e4e7640 1 -- 172.21.15.19:0/3349581410 wait complete. 2024-04-25T10:55:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.222+0000 7fb21e4e7640 1 Processor -- start 2024-04-25T10:55:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.222+0000 7fb21e4e7640 1 -- start start 2024-04-25T10:55:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21e4e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2181031b0 0x7fb21819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21e4e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb218108a50 0x7fb21819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21e4e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb21819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21e4e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb218112cc0 con 0x7fb218108ef0 2024-04-25T10:55:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21e4e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb218112b40 con 0x7fb218108a50 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21e4e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb218112e40 con 0x7fb2181031b0 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21ca84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb21819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb217fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2181031b0 0x7fb21819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21ca84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb21819c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51226/0 (socket says 172.21.15.19:51226) 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb21ca84640 1 -- 172.21.15.19:0/2794627786 learned_addr learned my addr 172.21.15.19:0/2794627786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.223+0000 7fb2177fe640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb218108a50 0x7fb21819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21ca84640 1 -- 172.21.15.19:0/2794627786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2181031b0 msgr2=0x7fb21819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21ca84640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2181031b0 0x7fb21819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21ca84640 1 -- 172.21.15.19:0/2794627786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb218108a50 msgr2=0x7fb21819bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21ca84640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb218108a50 0x7fb21819bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21ca84640 1 -- 172.21.15.19:0/2794627786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2181a6930 con 0x7fb218108ef0 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb217fff640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2181031b0 0x7fb21819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21ca84640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb21819c0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb21000eb10 tx=0x7fb21000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb2157fa640 1 -- 172.21.15.19:0/2794627786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210004b20 con 0x7fb218108ef0 2024-04-25T10:55:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb2157fa640 1 -- 172.21.15.19:0/2794627786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb210004cc0 con 0x7fb218108ef0 2024-04-25T10:55:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb2157fa640 1 -- 172.21.15.19:0/2794627786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210010670 con 0x7fb218108ef0 2024-04-25T10:55:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2181a6c20 con 0x7fb218108ef0 2024-04-25T10:55:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.224+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2181a70f0 con 0x7fb218108ef0 2024-04-25T10:55:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.228+0000 7fb225397740 1 Processor -- start 2024-04-25T10:55:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.228+0000 7fb2157fa640 1 -- 172.21.15.19:0/2794627786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7fb210005020 con 0x7fb218108ef0 2024-04-25T10:55:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.228+0000 7fb225397740 1 -- start start 2024-04-25T10:55:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb225397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 0x564ae5c96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb225397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5c97030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb225397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58d64d0 0x564ae5c97570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb225397740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ae5bfd460 con 0x564ae58d64d0 2024-04-25T10:55:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb225397740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564ae5bb7590 con 0x564ae58ce2f0 2024-04-25T10:55:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb225397740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ae5bf9420 con 0x564ae58f71b0 2024-04-25T10:55:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58d64d0 0x564ae5c97570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58d64d0 0x564ae5c97570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51238/0 (socket says 172.21.15.19:51238) 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb2177fe640 1 -- 172.21.15.19:0/3740567616 learned_addr learned my addr 172.21.15.19:0/3740567616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb217fff640 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5c97030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.229+0000 7fb2157fa640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb2000afd70 0x7fb2000b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb217fff640 1 -- 172.21.15.19:0/3740567616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 msgr2=0x564ae5c96af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb217fff640 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 0x564ae5c96af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb217fff640 1 -- 172.21.15.19:0/3740567616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58d64d0 msgr2=0x564ae5c97570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb217fff640 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58d64d0 0x564ae5c97570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb21ca84640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb2000afd70 0x7fb2000b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb217fff640 1 -- 172.21.15.19:0/3740567616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae5bc40d0 con 0x564ae58ce2f0 2024-04-25T10:55:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb2177fe640 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58d64d0 0x564ae5c97570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb2157fa640 1 -- 172.21.15.19:0/2794627786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb2100df870 con 0x7fb218108ef0 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb217fff640 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5c97030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb218103740 tx=0x7fb2080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=713a5b9117bb674d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb1fe7fc640 1 -- 172.21.15.19:0/3740567616 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208004220 con 0x564ae58ce2f0 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb1fe7fc640 1 -- 172.21.15.19:0/3740567616 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2080043c0 con 0x564ae58ce2f0 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.230+0000 7fb1fe7fc640 1 -- 172.21.15.19:0/3740567616 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20800f910 con 0x564ae58ce2f0 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 -- 172.21.15.19:0/3740567616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 msgr2=0x564ae5c97030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb21ca84640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb2000afd70 0x7fb2000b2280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2100dced0 tx=0x7fb21006e160 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5c97030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb218103740 tx=0x7fb2080118d0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 -- 172.21.15.19:0/3740567616 shutdown_connections 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58d64d0 0x564ae5c97570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5c97030 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 --2- 172.21.15.19:0/3740567616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 0x564ae5c96af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 -- 172.21.15.19:0/3740567616 >> 172.21.15.19:0/3740567616 conn(0x564ae58eb300 msgr2=0x564ae5c932a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 -- 172.21.15.19:0/3740567616 shutdown_connections 2024-04-25T10:55:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 -- 172.21.15.19:0/3740567616 wait complete. 2024-04-25T10:55:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.232+0000 7fb225397740 1 -- start start 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb225397740 1 Processor -- start 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb225397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5ca2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb225397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5ca30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb225397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 0x564ae5ca1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5ca2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5ca2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51242/0 (socket says 172.21.15.19:51242) 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb2177fe640 1 -- 172.21.15.19:0/2938463697 learned_addr learned my addr 172.21.15.19:0/2938463697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ae5ca3c20 con 0x564ae58ce2f0 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564ae5b84b60 con 0x564ae58d64d0 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.233+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ae5ca3da0 con 0x564ae58f71b0 2024-04-25T10:55:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb217fff640 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5ca30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb225397740 10 client.0 _ll_register_callbacks cb 0x564ae5ba92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb2177fe640 1 -- 172.21.15.19:0/2938463697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 msgr2=0x564ae5ca1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb2177fe640 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 0x564ae5ca1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb2177fe640 1 -- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 msgr2=0x564ae5ca30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb2177fe640 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5ca30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb2177fe640 1 -- 172.21.15.19:0/2938463697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae590f210 con 0x564ae58ce2f0 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb217fff640 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5ca30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb2177fe640 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5ca2b90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb20c02aa00 tx=0x7fb20c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80aad8d69e9017f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb1f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.234+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2080043c0 con 0x564ae58ce2f0 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb1f77fe640 1 -- 172.21.15.19:0/2938463697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c035040 con 0x564ae58ce2f0 2024-04-25T10:55:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb1f77fe640 1 -- 172.21.15.19:0/2938463697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb20c005450 con 0x564ae58ce2f0 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb1f77fe640 1 -- 172.21.15.19:0/2938463697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb20c004750 con 0x564ae58ce2f0 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb225397740 10 client.15678 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=20}) -- 0x564ae59db620 con 0x564ae58ce2f0 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb1f77fe640 1 -- 172.21.15.19:0/2938463697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb20c044070 con 0x564ae58ce2f0 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb225397740 10 client.15678 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb225397740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.235+0000 7fb225397740 20 client.15678 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.236+0000 7fb225397740 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58b5d80 0x564ae5cb35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.236+0000 7fb225397740 4 client.15678 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.236+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb2080043c0 con 0x564ae58b5d80 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.236+0000 7fb217fff640 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58b5d80 0x564ae5cb35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.236+0000 7fb217fff640 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58b5d80 0x564ae5cb35c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d2fa82b9308eae server_cookie=6007dfacc9bd981c in_seq=0 out_seq=0 2024-04-25T10:55:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.236+0000 7fb1f77fe640 10 client.15678 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.237+0000 7fb1f77fe640 1 -- 172.21.15.19:0/2938463697 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb2080043c0 con 0x564ae58b5d80 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.237+0000 7fb1f77fe640 10 client.15678 handle_command_reply: tid=0 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.237+0000 7fb225397740 1 client.15678 shutdown 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.237+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 msgr2=0x564ae5ca2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.237+0000 7fb225397740 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5ca2b90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb20c02aa00 tx=0x7fb20c004390 comp rx=0 tx=0).stop 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 shutdown_connections 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58b5d80 0x564ae5cb35c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58f71b0 0x564ae5ca1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb1f77fe640 0 client.15678 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5ca30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 --2- 172.21.15.19:0/2938463697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5ca2b90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 >> 172.21.15.19:0/2938463697 conn(0x564ae58eb300 msgr2=0x564ae5c98810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 shutdown_connections 2024-04-25T10:55:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 1 -- 172.21.15.19:0/2938463697 wait complete. 2024-04-25T10:55:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.238+0000 7fb225397740 20 client.15678 trim_cache size 0 max 16384 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 Processor -- start 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 -- start start 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58b5d80 0x564ae5c98e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5cb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5cb9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2181ada10 con 0x564ae58ce2f0 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2181b0240 con 0x564ae58d64d0 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb225397740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb2181ae030 con 0x564ae58b5d80 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb217fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5cb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58b5d80 0x564ae5c98e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb217fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5cb8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51258/0 (socket says 172.21.15.19:51258) 2024-04-25T10:55:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb217fff640 1 -- 172.21.15.19:0/204292101 learned_addr learned my addr 172.21.15.19:0/204292101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.269+0000 7fb2177fe640 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5cb9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb217fff640 1 -- 172.21.15.19:0/204292101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58b5d80 msgr2=0x564ae5c98e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb217fff640 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58b5d80 0x564ae5c98e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb217fff640 1 -- 172.21.15.19:0/204292101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 msgr2=0x564ae5cb9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb217fff640 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5cb9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb217fff640 1 -- 172.21.15.19:0/204292101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae5cb3b20 con 0x564ae58ce2f0 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb2177fe640 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58b5d80 0x564ae5c98e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb217fff640 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5cb8d50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb208013010 tx=0x7fb208005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e318f7126ba95c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb1fdffb640 1 -- 172.21.15.19:0/204292101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20802d2d0 con 0x564ae58ce2f0 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb1fdffb640 1 -- 172.21.15.19:0/204292101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb208032070 con 0x564ae58ce2f0 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.270+0000 7fb1fdffb640 1 -- 172.21.15.19:0/204292101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208003920 con 0x564ae58ce2f0 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.271+0000 7fb225397740 1 -- 172.21.15.19:0/204292101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 msgr2=0x564ae5cb8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.271+0000 7fb225397740 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5cb8d50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb208013010 tx=0x7fb208005f90 comp rx=0 tx=0).stop 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.271+0000 7fb225397740 1 -- 172.21.15.19:0/204292101 shutdown_connections 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58d64d0 0x564ae5cb9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58ce2f0 0x564ae5cb8d50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 --2- 172.21.15.19:0/204292101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58b5d80 0x564ae5c98e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 -- 172.21.15.19:0/204292101 >> 172.21.15.19:0/204292101 conn(0x564ae58eb300 msgr2=0x564ae5c98730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 -- 172.21.15.19:0/204292101 shutdown_connections 2024-04-25T10:55:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 -- 172.21.15.19:0/204292101 wait complete. 2024-04-25T10:55:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 -- start start 2024-04-25T10:55:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.272+0000 7fb225397740 1 Processor -- start 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb225397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58b5d80 0x564ae5ca9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb225397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5ca6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58b5d80 0x564ae5ca9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb225397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58d64d0 0x564ae5ca6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb225397740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2181ae700 con 0x564ae58b5d80 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb225397740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2181afb70 con 0x564ae58ce2f0 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb225397740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb2181aedd0 con 0x564ae58d64d0 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58b5d80 0x564ae5ca9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51262/0 (socket says 172.21.15.19:51262) 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb2177fe640 1 -- 172.21.15.19:0/3318994384 learned_addr learned my addr 172.21.15.19:0/3318994384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb225397740 10 client.0 _ll_register_callbacks cb 0x7fb20c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb217fff640 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5ca6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb2177fe640 1 -- 172.21.15.19:0/3318994384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58d64d0 msgr2=0x564ae5ca6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.273+0000 7fb2177fe640 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58d64d0 0x564ae5ca6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb2177fe640 1 -- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 msgr2=0x564ae5ca6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb2177fe640 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5ca6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb2177fe640 1 -- 172.21.15.19:0/3318994384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae5c99380 con 0x564ae58b5d80 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb2177fe640 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58b5d80 0x564ae5ca9420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009850 tx=0x7fb20c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8467294d0ff6aa9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb1f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb1f77fe640 1 -- 172.21.15.19:0/3318994384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c035040 con 0x564ae58b5d80 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb1f77fe640 1 -- 172.21.15.19:0/3318994384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb20c03a5f0 con 0x564ae58b5d80 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb208032070 con 0x564ae58b5d80 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb1f77fe640 1 -- 172.21.15.19:0/3318994384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb20c064050 con 0x564ae58b5d80 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb225397740 10 client.15684 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564ae5d1b480 con 0x564ae58b5d80 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.274+0000 7fb1f77fe640 1 -- 172.21.15.19:0/3318994384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20c03aca0 con 0x564ae58b5d80 2024-04-25T10:55:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.275+0000 7fb1f77fe640 1 -- 172.21.15.19:0/3318994384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb20c044070 con 0x564ae58b5d80 2024-04-25T10:55:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb225397740 10 client.15684 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb225397740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb225397740 20 client.15684 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb225397740 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58f71b0 0x564ae5d1dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb225397740 4 client.15684 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:55:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb208032070 con 0x564ae58f71b0 2024-04-25T10:55:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb217fff640 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58f71b0 0x564ae5d1dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb1f77fe640 10 client.15684 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.276+0000 7fb217fff640 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58f71b0 0x564ae5d1dc00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639256972a9dfb07 server_cookie=ff9ee402329f0890 in_seq=0 out_seq=0 2024-04-25T10:55:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb1f77fe640 1 -- 172.21.15.19:0/3318994384 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb208032070 con 0x564ae58f71b0 2024-04-25T10:55:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb1f77fe640 10 client.15684 handle_command_reply: tid=0 2024-04-25T10:55:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 client.15684 shutdown 2024-04-25T10:55:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58b5d80 msgr2=0x564ae5ca9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58b5d80 0x564ae5ca9420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009850 tx=0x7fb20c02a410 comp rx=0 tx=0).stop 2024-04-25T10:55:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 shutdown_connections 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564ae58f71b0 0x564ae5d1dc00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb1f77fe640 0 client.15684 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ae58d64d0 0x564ae5ca6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ae58ce2f0 0x564ae5ca6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.280+0000 7fb225397740 1 --2- 172.21.15.19:0/3318994384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ae58b5d80 0x564ae5ca9420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.281+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 >> 172.21.15.19:0/3318994384 conn(0x564ae58eb300 msgr2=0x564ae5ca2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.281+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 shutdown_connections 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.281+0000 7fb225397740 1 -- 172.21.15.19:0/3318994384 wait complete. 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.281+0000 7fb225397740 20 client.15684 trim_cache size 0 max 16384 2024-04-25T10:55:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:55:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:55:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:55:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:55:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.281+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb2000afd70 msgr2=0x7fb2000b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.281+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb2000afd70 0x7fb2000b2280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2100dced0 tx=0x7fb21006e160 comp rx=0 tx=0).stop 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.281+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 msgr2=0x7fb21819c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb21819c0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb21000eb10 tx=0x7fb21000c2d0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 shutdown_connections 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb2000afd70 0x7fb2000b2280 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb218108ef0 0x7fb21819c0a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb218108a50 0x7fb21819bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 --2- 172.21.15.19:0/2794627786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2181031b0 0x7fb21819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 >> 172.21.15.19:0/2794627786 conn(0x7fb2180fe260 msgr2=0x7fb2180fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 shutdown_connections 2024-04-25T10:55:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:42.282+0000 7fb21e4e7640 1 -- 172.21.15.19:0/2794627786 wait complete. 2024-04-25T10:55:42.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:42 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:55:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:42 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:55:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:42 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:55:43.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:43 smithi019 ceph-mon[29209]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:43 smithi169 ceph-mon[33049]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:43.561 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:43 smithi112 ceph-mon[32799]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:45.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:45 smithi019 ceph-mon[29209]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-25T10:55:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:45 smithi169 ceph-mon[33049]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-25T10:55:45.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:45 smithi112 ceph-mon[32799]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-25T10:55:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:47 smithi019 ceph-mon[29209]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:47 smithi169 ceph-mon[33049]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:47.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:47 smithi112 ceph-mon[32799]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:49 smithi019 ceph-mon[29209]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:49 smithi169 ceph-mon[33049]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:49 smithi112 ceph-mon[32799]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:51.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:51 smithi019 ceph-mon[29209]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:51 smithi169 ceph-mon[33049]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:51.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:51 smithi112 ceph-mon[32799]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:53 smithi019 ceph-mon[29209]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:53 smithi169 ceph-mon[33049]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:53.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:53 smithi112 ceph-mon[32799]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T10:55:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 -- 172.21.15.19:0/2350789888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 msgr2=0x7f3488102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 --2- 172.21.15.19:0/2350789888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f3488102b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f347c009900 tx=0x7f347c02a410 comp rx=0 tx=0).stop 2024-04-25T10:55:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 -- 172.21.15.19:0/2350789888 shutdown_connections 2024-04-25T10:55:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 --2- 172.21.15.19:0/2350789888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34881031c0 0x7f348810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 --2- 172.21.15.19:0/2350789888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f3488102b50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 --2- 172.21.15.19:0/2350789888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3488108a60 0x7f3488102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 -- 172.21.15.19:0/2350789888 >> 172.21.15.19:0/2350789888 conn(0x7f34880781b0 msgr2=0x7f34880785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 -- 172.21.15.19:0/2350789888 shutdown_connections 2024-04-25T10:55:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.431+0000 7f348d692640 1 -- 172.21.15.19:0/2350789888 wait complete. 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.432+0000 7f348d692640 1 Processor -- start 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.432+0000 7f348d692640 1 -- start start 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.432+0000 7f348d692640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34881031c0 0x7f348819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.432+0000 7f348d692640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3488108a60 0x7f348819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f348d692640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f348819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f348d692640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3488112c80 con 0x7f3488108f00 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f348d692640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3488112b00 con 0x7f34881031c0 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f348d692640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3488112e00 con 0x7f3488108a60 2024-04-25T10:55:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f348819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f348819c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46608/0 (socket says 172.21.15.19:46608) 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 -- 172.21.15.19:0/1385944972 learned_addr learned my addr 172.21.15.19:0/1385944972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34867fc640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3488108a60 0x7f348819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34881031c0 0x7f348819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 -- 172.21.15.19:0/1385944972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3488108a60 msgr2=0x7f348819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3488108a60 0x7f348819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 -- 172.21.15.19:0/1385944972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34881031c0 msgr2=0x7f348819b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34881031c0 0x7f348819b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f34877fe640 1 -- 172.21.15.19:0/1385944972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34881a6940 con 0x7f3488108f00 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.433+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34881031c0 0x7f348819b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.434+0000 7f34877fe640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f348819c0a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f348000eb10 tx=0x7f348000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.434+0000 7f346ffff640 1 -- 172.21.15.19:0/1385944972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480004b20 con 0x7f3488108f00 2024-04-25T10:55:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.434+0000 7f346ffff640 1 -- 172.21.15.19:0/1385944972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3480004cc0 con 0x7f3488108f00 2024-04-25T10:55:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.434+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3488199450 con 0x7f3488108f00 2024-04-25T10:55:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.434+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3488199910 con 0x7f3488108f00 2024-04-25T10:55:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.435+0000 7f346ffff640 1 -- 172.21.15.19:0/1385944972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480010670 con 0x7f3488108f00 2024-04-25T10:55:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.438+0000 7f349461e740 1 Processor -- start 2024-04-25T10:55:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.438+0000 7f346ffff640 1 -- 172.21.15.19:0/1385944972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f34800108f0 con 0x7f3488108f00 2024-04-25T10:55:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.438+0000 7f349461e740 1 -- start start 2024-04-25T10:55:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f349461e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 0x558feb1fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f349461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f349461e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb1fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f349461e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558feb163460 con 0x558feae3c4d0 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f349461e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558feb11d590 con 0x558feae342f0 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f349461e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558feb15f420 con 0x558feae5d1b0 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f3486ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f34867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb1fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f3486ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:37816/0 (socket says 172.21.15.19:37816) 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f3486ffd640 1 -- 172.21.15.19:0/1144550623 learned_addr learned my addr 172.21.15.19:0/1144550623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f346ffff640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f34700afdf0 0x7f34700b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f3486ffd640 1 -- 172.21.15.19:0/1144550623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 msgr2=0x558feb1fcaf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 0x558feb1fcaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f3486ffd640 1 -- 172.21.15.19:0/1144550623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 msgr2=0x558feb1fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.439+0000 7f34877fe640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f34700afdf0 0x7f34700b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb1fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f3486ffd640 1 -- 172.21.15.19:0/1144550623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558feb12a0d0 con 0x558feae342f0 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f346ffff640 1 -- 172.21.15.19:0/1385944972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f34800df710 con 0x7f3488108f00 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f34867fc640 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb1fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fd030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3478009610 tx=0x7f3478009ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f4fbe2d5b5faa04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f34877fe640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f34700afdf0 0x7f34700b2300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f34800dcd70 tx=0x7f348001bd20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f346d7fa640 1 -- 172.21.15.19:0/1144550623 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478005020 con 0x558feae342f0 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f346d7fa640 1 -- 172.21.15.19:0/1144550623 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34780047a0 con 0x558feae342f0 2024-04-25T10:55:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.440+0000 7f346d7fa640 1 -- 172.21.15.19:0/1144550623 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780088b0 con 0x558feae342f0 2024-04-25T10:55:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.441+0000 7f349461e740 1 -- 172.21.15.19:0/1144550623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 msgr2=0x558feb1fd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.441+0000 7f349461e740 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fd030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3478009610 tx=0x7f3478009ad0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 -- 172.21.15.19:0/1144550623 shutdown_connections 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb1fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fd030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 --2- 172.21.15.19:0/1144550623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 0x558feb1fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 -- 172.21.15.19:0/1144550623 >> 172.21.15.19:0/1144550623 conn(0x558feae51300 msgr2=0x558feb1f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 -- 172.21.15.19:0/1144550623 shutdown_connections 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 -- 172.21.15.19:0/1144550623 wait complete. 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.442+0000 7f349461e740 1 -- start start 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.443+0000 7f349461e740 1 Processor -- start 2024-04-25T10:55:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.443+0000 7f349461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb2071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.443+0000 7f349461e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb207710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.443+0000 7f349461e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 0x558feb20d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f349461e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558feb209c20 con 0x558feae3c4d0 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f3486ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb207710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f34867fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb2071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f3486ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb207710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46628/0 (socket says 172.21.15.19:46628) 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f3486ffd640 1 -- 172.21.15.19:0/1989345480 learned_addr learned my addr 172.21.15.19:0/1989345480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558feb0eab60 con 0x558feae342f0 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558feb209da0 con 0x558feae5d1b0 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f34867fc640 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 0x558feb20d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f34867fc640 1 -- 172.21.15.19:0/1989345480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 msgr2=0x558feb20d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f34867fc640 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 0x558feb20d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f34867fc640 1 -- 172.21.15.19:0/1989345480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 msgr2=0x558feb207710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f34867fc640 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb207710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f34867fc640 1 -- 172.21.15.19:0/1989345480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558feae75210 con 0x558feae342f0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb207710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.444+0000 7f349461e740 10 client.0 _ll_register_callbacks cb 0x558feb10f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f34867fc640 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb2071b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f347c02a930 tx=0x7f347c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc0cd9ec2f63a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f346aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34780047a0 con 0x558feae342f0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f346aff5640 1 -- 172.21.15.19:0/1989345480 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c035040 con 0x558feae342f0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f346aff5640 1 -- 172.21.15.19:0/1989345480 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f347c031b10 con 0x558feae342f0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f346aff5640 1 -- 172.21.15.19:0/1989345480 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f347c031d70 con 0x558feae342f0 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f349461e740 10 client.25382 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.445+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=20}) -- 0x558feaf41620 con 0x558feae342f0 2024-04-25T10:55:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f346aff5640 1 -- 172.21.15.19:0/1989345480 <== mon.2 v2:172.21.15.112:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f347c004030 con 0x558feae342f0 2024-04-25T10:55:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f349461e740 10 client.25382 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f349461e740 10 client.25382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f349461e740 20 client.25382 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f349461e740 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae1bd80 0x558feb219610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f349461e740 4 client.25382 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f34780047a0 con 0x558feae1bd80 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.446+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae1bd80 0x558feb219610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.447+0000 7f346aff5640 10 client.25382 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.447+0000 7f3486ffd640 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae1bd80 0x558feb219610 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3c9457956e118b server_cookie=b1c2a422fc441ea5 in_seq=0 out_seq=0 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.447+0000 7f346aff5640 1 -- 172.21.15.19:0/1989345480 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f34780047a0 con 0x558feae1bd80 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.447+0000 7f346aff5640 10 client.25382 handle_command_reply: tid=0 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.447+0000 7f349461e740 1 client.25382 shutdown 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 msgr2=0x558feb2071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb2071b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f347c02a930 tx=0x7f347c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 shutdown_connections 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae1bd80 0x558feb219610 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae5d1b0 0x558feb20d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f346aff5640 0 client.25382 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb207710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 --2- 172.21.15.19:0/1989345480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb2071b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 >> 172.21.15.19:0/1989345480 conn(0x558feae51300 msgr2=0x558feb1fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 shutdown_connections 2024-04-25T10:55:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 1 -- 172.21.15.19:0/1989345480 wait complete. 2024-04-25T10:55:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.448+0000 7f349461e740 20 client.25382 trim_cache size 0 max 16384 2024-04-25T10:55:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.479+0000 7f349461e740 1 Processor -- start 2024-04-25T10:55:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.479+0000 7f349461e740 1 -- start start 2024-04-25T10:55:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f349461e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae1bd80 0x558feb20d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f349461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f349461e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb21ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f349461e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f347400c620 con 0x558feae3c4d0 2024-04-25T10:55:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f349461e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f347400eea0 con 0x558feae342f0 2024-04-25T10:55:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f349461e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f347400cc90 con 0x558feae1bd80 2024-04-25T10:55:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f34867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb21ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f34867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb21ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46640/0 (socket says 172.21.15.19:46640) 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f34867fc640 1 -- 172.21.15.19:0/3949224796 learned_addr learned my addr 172.21.15.19:0/3949224796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.480+0000 7f34867fc640 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae1bd80 0x558feb20d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f3486ffd640 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f34867fc640 1 -- 172.21.15.19:0/3949224796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae1bd80 msgr2=0x558feb20d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f34867fc640 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae1bd80 0x558feb20d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f34867fc640 1 -- 172.21.15.19:0/3949224796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 msgr2=0x558feb1fee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f34867fc640 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f34867fc640 1 -- 172.21.15.19:0/3949224796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558feb20de50 con 0x558feae3c4d0 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f34867fc640 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb21ee10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f347c001eb0 tx=0x7f347c03f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229f67761405f0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f346cff9640 1 -- 172.21.15.19:0/3949224796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c000e60 con 0x558feae3c4d0 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f346cff9640 1 -- 172.21.15.19:0/3949224796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f347c030850 con 0x558feae3c4d0 2024-04-25T10:55:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.481+0000 7f346cff9640 1 -- 172.21.15.19:0/3949224796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c035070 con 0x558feae3c4d0 2024-04-25T10:55:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 -- 172.21.15.19:0/3949224796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 msgr2=0x558feb21ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb21ee10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f347c001eb0 tx=0x7f347c03f070 comp rx=0 tx=0).stop 2024-04-25T10:55:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 -- 172.21.15.19:0/3949224796 shutdown_connections 2024-04-25T10:55:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae3c4d0 0x558feb21ee10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae342f0 0x558feb1fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 --2- 172.21.15.19:0/3949224796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae1bd80 0x558feb20d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 -- 172.21.15.19:0/3949224796 >> 172.21.15.19:0/3949224796 conn(0x558feae51300 msgr2=0x558feb1fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 -- 172.21.15.19:0/3949224796 shutdown_connections 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 -- 172.21.15.19:0/3949224796 wait complete. 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.483+0000 7f349461e740 1 -- start start 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 1 Processor -- start 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae1bd80 0x558feb279cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae342f0 0x558feb27a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae3c4d0 0x558feb20ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f347400d360 con 0x558feae1bd80 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f347400e7d0 con 0x558feae3c4d0 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f347400da30 con 0x558feae342f0 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.484+0000 7f349461e740 10 client.0 _ll_register_callbacks cb 0x7f347c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f34867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae1bd80 0x558feb279cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f3486ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae342f0 0x558feb27a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f34867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae1bd80 0x558feb279cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46652/0 (socket says 172.21.15.19:46652) 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f34867fc640 1 -- 172.21.15.19:0/257351014 learned_addr learned my addr 172.21.15.19:0/257351014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f3486ffd640 1 -- 172.21.15.19:0/257351014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae3c4d0 msgr2=0x558feb20ecf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f3486ffd640 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae3c4d0 0x558feb20ecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f3486ffd640 1 -- 172.21.15.19:0/257351014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae1bd80 msgr2=0x558feb279cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f3486ffd640 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae1bd80 0x558feb279cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f3486ffd640 1 -- 172.21.15.19:0/257351014 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558feb20f300 con 0x558feae342f0 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f34867fc640 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae1bd80 0x558feb279cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f3486ffd640 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae342f0 0x558feb27a200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x558feb1f3690 tx=0x7f3478008530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec8d8fbcd186022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.485+0000 7f346aff5640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.486+0000 7f346aff5640 1 -- 172.21.15.19:0/257351014 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478031030 con 0x558feae342f0 2024-04-25T10:55:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.486+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f347c030850 con 0x558feae342f0 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.486+0000 7f346aff5640 1 -- 172.21.15.19:0/257351014 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f347801f040 con 0x558feae342f0 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.486+0000 7f346aff5640 1 -- 172.21.15.19:0/257351014 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3478005250 con 0x558feae342f0 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.486+0000 7f349461e740 10 client.25306 fetch_fsmap learned FSMap version 20 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.486+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558feb2812c0 con 0x558feae342f0 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.486+0000 7f346aff5640 1 -- 172.21.15.19:0/257351014 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f347c030850 con 0x558feae342f0 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.487+0000 7f346aff5640 1 -- 172.21.15.19:0/257351014 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f347802d4c0 con 0x558feae342f0 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.487+0000 7f349461e740 10 client.25306 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.487+0000 7f349461e740 10 client.25306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.487+0000 7f349461e740 20 client.25306 populate_metadata read hostname 'smithi019' 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.487+0000 7f349461e740 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae5d1b0 0x558feb283800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.487+0000 7f349461e740 4 client.25306 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:55:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.487+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558feb171070 con 0x558feae5d1b0 2024-04-25T10:55:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.488+0000 7f34867fc640 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae5d1b0 0x558feb283800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:55:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.488+0000 7f346aff5640 10 client.25306 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.488+0000 7f34867fc640 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae5d1b0 0x558feb283800 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302e44059a2ff3d server_cookie=f2796877379123e6 in_seq=0 out_seq=0 2024-04-25T10:55:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.492+0000 7f346aff5640 1 -- 172.21.15.19:0/257351014 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558feb171070 con 0x558feae5d1b0 2024-04-25T10:55:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.492+0000 7f346aff5640 10 client.25306 handle_command_reply: tid=0 2024-04-25T10:55:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.492+0000 7f349461e740 1 client.25306 shutdown 2024-04-25T10:55:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae342f0 msgr2=0x558feb27a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae342f0 0x558feb27a200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x558feb1f3690 tx=0x7f3478008530 comp rx=0 tx=0).stop 2024-04-25T10:55:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 shutdown_connections 2024-04-25T10:55:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558feae5d1b0 0x558feb283800 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558feae3c4d0 0x558feb20ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f346aff5640 0 client.25306 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558feae342f0 0x558feb27a200 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 --2- 172.21.15.19:0/257351014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558feae1bd80 0x558feb279cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 >> 172.21.15.19:0/257351014 conn(0x558feae51300 msgr2=0x558feb1fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 shutdown_connections 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 1 -- 172.21.15.19:0/257351014 wait complete. 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.493+0000 7f349461e740 20 client.25306 trim_cache size 0 max 16384 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:55:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:55:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:55:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:55:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f34700afdf0 msgr2=0x7f34700b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f34700afdf0 0x7f34700b2300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f34800dcd70 tx=0x7f348001bd20 comp rx=0 tx=0).stop 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 msgr2=0x7f348819c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f348819c0a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f348000eb10 tx=0x7f348000c2d0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 shutdown_connections 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f34700afdf0 0x7f34700b2300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3488108f00 0x7f348819c0a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3488108a60 0x7f348819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 --2- 172.21.15.19:0/1385944972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34881031c0 0x7f348819b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 >> 172.21.15.19:0/1385944972 conn(0x7f34880781b0 msgr2=0x7f3488104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 shutdown_connections 2024-04-25T10:55:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:55:55.494+0000 7f348d692640 1 -- 172.21.15.19:0/1385944972 wait complete. 2024-04-25T10:55:55.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:55 smithi169 ceph-mon[33049]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:55.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:55 smithi112 ceph-mon[32799]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:55.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:55 smithi019 ceph-mon[29209]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T10:55:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:57 smithi169 ceph-mon[33049]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T10:55:57.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:57 smithi112 ceph-mon[32799]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T10:55:57.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:57 smithi019 ceph-mon[29209]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T10:55:59.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:55:59 smithi169 ceph-mon[33049]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:55:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:55:59 smithi112 ceph-mon[32799]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:55:59.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:55:59 smithi019 ceph-mon[29209]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:56:00.014 INFO:tasks.workunit.client.0.smithi019.stderr:Updating files: 93% (11481/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-25T10:56:00.319 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T10:56:00.319 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-25T10:56:00.319 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-04-25T10:56:00.320 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-25T10:56:00.321 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-04-25T10:56:00.321 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-25T10:56:00.321 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T10:56:00.321 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-25T10:56:00.321 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:56:00.326 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T10:56:00.389 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T10:56:00.391 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T10:56:00.391 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T10:56:01.027 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T10:56:01.083 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T10:56:01.124 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T10:56:01.126 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T10:56:01.126 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T10:56:01.165 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T10:56:01.169 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:56:01.169 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T10:56:01.225 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-04-25T10:56:01.227 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-04-25T10:56:01.227 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-04-25T10:56:01.286 INFO:tasks.workunit.client.0.smithi019.stdout:test large (16MB) dio write 2024-04-25T10:56:01.287 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test large (16MB) dio write' 2024-04-25T10:56:01.287 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-04-25T10:56:01.389 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records in 2024-04-25T10:56:01.389 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records out 2024-04-25T10:56:01.389 INFO:tasks.workunit.client.0.smithi019.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.100513 s, 167 MB/s 2024-04-25T10:56:01.389 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo OK 2024-04-25T10:56:01.390 INFO:tasks.workunit.client.0.smithi019.stdout:OK 2024-04-25T10:56:01.390 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T10:56:01.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T10:56:01.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:01 smithi019 ceph-mon[29209]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:56:01.463 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-04-25T10:56:01.464 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-04-25T10:56:01.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:01 smithi169 ceph-mon[33049]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:56:01.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:01 smithi112 ceph-mon[32799]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T10:56:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:03 smithi169 ceph-mon[33049]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-25T10:56:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:03 smithi169 ceph-mon[33049]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T10:56:03.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:03 smithi112 ceph-mon[32799]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-25T10:56:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:03 smithi112 ceph-mon[32799]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T10:56:03.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:03 smithi019 ceph-mon[29209]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-25T10:56:03.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:03 smithi019 ceph-mon[29209]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T10:56:05.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:05 smithi169 ceph-mon[33049]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-25T10:56:05.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:05 smithi112 ceph-mon[32799]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-25T10:56:05.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:05 smithi019 ceph-mon[29209]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-25T10:56:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:07 smithi169 ceph-mon[33049]: pgmap v51: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:07.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:07 smithi112 ceph-mon[32799]: pgmap v51: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:07 smithi019 ceph-mon[29209]: pgmap v51: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.597+0000 7f820973c640 1 -- 172.21.15.19:0/315748259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82041031d0 msgr2=0x7f820410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.597+0000 7f820973c640 1 --2- 172.21.15.19:0/315748259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82041031d0 0x7f820410f5a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009900 tx=0x7f81fc02a410 comp rx=0 tx=0).stop 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 -- 172.21.15.19:0/315748259 shutdown_connections 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 --2- 172.21.15.19:0/315748259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82041031d0 0x7f820410f5a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 --2- 172.21.15.19:0/315748259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8204108f20 0x7f8204102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 --2- 172.21.15.19:0/315748259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8204108a80 0x7f8204102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 -- 172.21.15.19:0/315748259 >> 172.21.15.19:0/315748259 conn(0x7f82040781b0 msgr2=0x7f82040785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 -- 172.21.15.19:0/315748259 shutdown_connections 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 -- 172.21.15.19:0/315748259 wait complete. 2024-04-25T10:56:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.598+0000 7f820973c640 1 Processor -- start 2024-04-25T10:56:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f820973c640 1 -- start start 2024-04-25T10:56:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f820973c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82041031d0 0x7f820419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f820973c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8204108a80 0x7f820419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f820973c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8204108f20 0x7f820419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f820973c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8204112c70 con 0x7f8204108f20 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f820973c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8204112af0 con 0x7f8204108a80 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f820973c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8204112df0 con 0x7f82041031d0 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8204108f20 0x7f820419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8204108f20 0x7f820419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50112/0 (socket says 172.21.15.19:50112) 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82041031d0 0x7f820419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 -- 172.21.15.19:0/3435109750 learned_addr learned my addr 172.21.15.19:0/3435109750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8204108a80 0x7f820419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 -- 172.21.15.19:0/3435109750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82041031d0 msgr2=0x7f820419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82041031d0 0x7f820419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 -- 172.21.15.19:0/3435109750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8204108a80 msgr2=0x7f820419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8204108a80 0x7f820419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 -- 172.21.15.19:0/3435109750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820419f5a0 con 0x7f8204108f20 2024-04-25T10:56:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.599+0000 7f82037fe640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8204108f20 0x7f820419ef60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004770 tx=0x7f81fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.600+0000 7f8200ff9640 1 -- 172.21.15.19:0/3435109750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc03f030 con 0x7f8204108f20 2024-04-25T10:56:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.600+0000 7f8200ff9640 1 -- 172.21.15.19:0/3435109750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81fc003aa0 con 0x7f8204108f20 2024-04-25T10:56:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.600+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820419f830 con 0x7f8204108f20 2024-04-25T10:56:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.600+0000 7f8200ff9640 1 -- 172.21.15.19:0/3435109750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc035070 con 0x7f8204108f20 2024-04-25T10:56:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.600+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820419fd20 con 0x7f8204108f20 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.602+0000 7f8200ff9640 1 -- 172.21.15.19:0/3435109750 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f81fc042030 con 0x7f8204108f20 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.602+0000 7f82105ea740 1 Processor -- start 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.602+0000 7f82105ea740 1 -- start start 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f82105ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f82105ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b35c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f82105ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af9b4d0 0x55fc5b35c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f82105ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc5b2c2460 con 0x55fc5afbc1b0 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f82105ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fc5b27c590 con 0x55fc5af9b4d0 2024-04-25T10:56:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f82105ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc5b2be420 con 0x55fc5af932f0 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8200ff9640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f81f00afc50 0x7f81f00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f81fa7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b35c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b35c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:51528/0 (socket says 172.21.15.19:51528) 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 -- 172.21.15.19:0/615645216 learned_addr learned my addr 172.21.15.19:0/615645216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f82037fe640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f81f00afc50 0x7f81f00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af9b4d0 0x55fc5b35c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 -- 172.21.15.19:0/615645216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af9b4d0 msgr2=0x55fc5b35c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af9b4d0 0x55fc5b35c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 -- 172.21.15.19:0/615645216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5afbc1b0 msgr2=0x55fc5b35baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8200ff9640 1 -- 172.21.15.19:0/3435109750 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f81fc0ff630 con 0x7f8204108f20 2024-04-25T10:56:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f8202ffd640 1 -- 172.21.15.19:0/615645216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5b2890d0 con 0x55fc5af932f0 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.603+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af9b4d0 0x55fc5b35c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.604+0000 7f8202ffd640 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b35c030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81f40039f0 tx=0x7f81f4003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e39d53814de0da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.604+0000 7f82037fe640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f81f00afc50 0x7f81f00b2160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0c38d0 tx=0x7f81fc0b0040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.604+0000 7f81f97fa640 1 -- 172.21.15.19:0/615645216 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4011440 con 0x55fc5af932f0 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.604+0000 7f81f97fa640 1 -- 172.21.15.19:0/615645216 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f4015030 con 0x55fc5af932f0 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.604+0000 7f81f97fa640 1 -- 172.21.15.19:0/615645216 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4010910 con 0x55fc5af932f0 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 -- 172.21.15.19:0/615645216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 msgr2=0x55fc5b35c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b35c030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81f40039f0 tx=0x7f81f4003eb0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 -- 172.21.15.19:0/615645216 shutdown_connections 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af9b4d0 0x55fc5b35c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b35c030 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 --2- 172.21.15.19:0/615645216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 -- 172.21.15.19:0/615645216 >> 172.21.15.19:0/615645216 conn(0x55fc5afb0300 msgr2=0x55fc5b3582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 -- 172.21.15.19:0/615645216 shutdown_connections 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 -- 172.21.15.19:0/615645216 wait complete. 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.605+0000 7f82105ea740 1 -- start start 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 1 Processor -- start 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b366190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b3666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b36c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc5b368c20 con 0x55fc5af9b4d0 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fc5b249b60 con 0x55fc5af932f0 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc5b368da0 con 0x55fc5afbc1b0 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f82105ea740 10 client.0 _ll_register_callbacks cb 0x55fc5b26e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b3666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b3666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50122/0 (socket says 172.21.15.19:50122) 2024-04-25T10:56:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 -- 172.21.15.19:0/509936325 learned_addr learned my addr 172.21.15.19:0/509936325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b36c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b366190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 -- 172.21.15.19:0/509936325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 msgr2=0x55fc5b36c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b36c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 -- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 msgr2=0x55fc5b366190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b366190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.606+0000 7f8202ffd640 1 -- 172.21.15.19:0/509936325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5afd4210 con 0x55fc5af9b4d0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f8202ffd640 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b3666d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f81f40105b0 tx=0x7f81f4005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b483cc429fa1d913 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f81ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f81ea7fc640 1 -- 172.21.15.19:0/509936325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4029060 con 0x55fc5af9b4d0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f81ea7fc640 1 -- 172.21.15.19:0/509936325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f81f4010860 con 0x55fc5af9b4d0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f4015030 con 0x55fc5af9b4d0 2024-04-25T10:56:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f81ea7fc640 1 -- 172.21.15.19:0/509936325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f81f4010a60 con 0x55fc5af9b4d0 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 10 client.15714 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55fc5b0a0620 con 0x55fc5af9b4d0 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f81ea7fc640 1 -- 172.21.15.19:0/509936325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f4015030 con 0x55fc5af9b4d0 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f81ea7fc640 1 -- 172.21.15.19:0/509936325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f81f4023070 con 0x55fc5af9b4d0 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 10 client.15714 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 20 client.15714 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af7ad80 0x55fc5b378530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 4 client.15714 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.607+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5b2d0070 con 0x55fc5af7ad80 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.608+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af7ad80 0x55fc5b378530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.608+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af7ad80 0x55fc5b378530 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f411c0fba306977 server_cookie=63af8d52a611bfd8 in_seq=0 out_seq=0 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.608+0000 7f81ea7fc640 10 client.15714 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f81ea7fc640 1 -- 172.21.15.19:0/509936325 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc5b2d0070 con 0x55fc5af7ad80 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f81ea7fc640 10 client.15714 handle_command_reply: tid=0 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 client.15714 shutdown 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 msgr2=0x55fc5b3666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b3666d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f81f40105b0 tx=0x7f81f4005ae0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 shutdown_connections 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af7ad80 0x55fc5b378530 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b36c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b3666d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 --2- 172.21.15.19:0/509936325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b366190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 >> 172.21.15.19:0/509936325 conn(0x55fc5afb0300 msgr2=0x55fc5b35d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f81ea7fc640 0 client.15714 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 shutdown_connections 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 1 -- 172.21.15.19:0/509936325 wait complete. 2024-04-25T10:56:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.609+0000 7f82105ea740 20 client.15714 trim_cache size 0 max 16384 2024-04-25T10:56:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 Processor -- start 2024-04-25T10:56:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 -- start start 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b36c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b358370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82041ad9c0 con 0x55fc5af9b4d0 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82041b0240 con 0x55fc5afbc1b0 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f82105ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82041ae030 con 0x55fc5af932f0 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b358370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f81fa7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b36c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b358370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50128/0 (socket says 172.21.15.19:50128) 2024-04-25T10:56:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.625+0000 7f8202ffd640 1 -- 172.21.15.19:0/2947527754 learned_addr learned my addr 172.21.15.19:0/2947527754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81fa7fc640 1 -- 172.21.15.19:0/2947527754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5afbc1b0 msgr2=0x55fc5b35f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81fa7fc640 1 -- 172.21.15.19:0/2947527754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 msgr2=0x55fc5b358370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b358370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81fa7fc640 1 -- 172.21.15.19:0/2947527754 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5b3591c0 con 0x55fc5af932f0 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f8202ffd640 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b358370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b36c010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55fc5b358c60 tx=0x7f81ec004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=389606e2a0e4e3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81f8ff9640 1 -- 172.21.15.19:0/2947527754 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec02c070 con 0x55fc5af932f0 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81f8ff9640 1 -- 172.21.15.19:0/2947527754 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ec0249b0 con 0x55fc5af932f0 2024-04-25T10:56:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.626+0000 7f81f8ff9640 1 -- 172.21.15.19:0/2947527754 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec003c60 con 0x55fc5af932f0 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 -- 172.21.15.19:0/2947527754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 msgr2=0x55fc5b36c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b36c010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55fc5b358c60 tx=0x7f81ec004730 comp rx=0 tx=0).stop 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 -- 172.21.15.19:0/2947527754 shutdown_connections 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5afbc1b0 0x55fc5b35f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b358370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 --2- 172.21.15.19:0/2947527754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5af932f0 0x55fc5b36c010 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 -- 172.21.15.19:0/2947527754 >> 172.21.15.19:0/2947527754 conn(0x55fc5afb0300 msgr2=0x55fc5b357c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 -- 172.21.15.19:0/2947527754 shutdown_connections 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 -- 172.21.15.19:0/2947527754 wait complete. 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.627+0000 7f82105ea740 1 -- start start 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 1 Processor -- start 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b36dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b36e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b3d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82041ae700 con 0x55fc5af9b4d0 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82041afb70 con 0x55fc5af932f0 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82041aedd0 con 0x55fc5afbc1b0 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f82105ea740 10 client.0 _ll_register_callbacks cb 0x7f81ec0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b36e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b36e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50130/0 (socket says 172.21.15.19:50130) 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 -- 172.21.15.19:0/1516855128 learned_addr learned my addr 172.21.15.19:0/1516855128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b3d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b36dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 -- 172.21.15.19:0/1516855128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 msgr2=0x55fc5b3d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b3d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 -- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 msgr2=0x55fc5b36dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b36dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 -- 172.21.15.19:0/1516855128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5b3d8bd0 con 0x55fc5af9b4d0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b36dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f8202ffd640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b36e300 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000f80 tx=0x7f81f40105e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818a6a0c46491be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.628+0000 7f81ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f81ea7fc640 1 -- 172.21.15.19:0/1516855128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4029040 con 0x55fc5af9b4d0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ec0249b0 con 0x55fc5af9b4d0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f81ea7fc640 1 -- 172.21.15.19:0/1516855128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f81f4010e10 con 0x55fc5af9b4d0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f81ea7fc640 1 -- 172.21.15.19:0/1516855128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f81f403d050 con 0x55fc5af9b4d0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 10 client.15720 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55fc5b3dfed0 con 0x55fc5af9b4d0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f81ea7fc640 1 -- 172.21.15.19:0/1516855128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ec0249b0 con 0x55fc5af9b4d0 2024-04-25T10:56:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f81ea7fc640 1 -- 172.21.15.19:0/1516855128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f81f4023070 con 0x55fc5af9b4d0 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 10 client.15720 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 20 client.15720 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af501c0 0x55fc5b3e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 4 client.15720 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.629+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc5b2c9ee0 con 0x55fc5af501c0 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.630+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af501c0 0x55fc5b3e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.630+0000 7f81ea7fc640 10 client.15720 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.630+0000 7f81fa7fc640 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af501c0 0x55fc5b3e26c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82f9f169c2f4a1f server_cookie=a5d99ea14d16f4d9 in_seq=0 out_seq=0 2024-04-25T10:56:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.640+0000 7f81ea7fc640 1 -- 172.21.15.19:0/1516855128 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc5b2c9ee0 con 0x55fc5af501c0 2024-04-25T10:56:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.640+0000 7f81ea7fc640 10 client.15720 handle_command_reply: tid=0 2024-04-25T10:56:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.640+0000 7f82105ea740 1 client.15720 shutdown 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.640+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 msgr2=0x55fc5b36e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.640+0000 7f82105ea740 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b36e300 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000f80 tx=0x7f81f40105e0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.640+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 shutdown_connections 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.640+0000 7f82105ea740 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55fc5af501c0 0x55fc5b3e26c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f82105ea740 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc5afbc1b0 0x55fc5b3d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f81ea7fc640 0 client.15720 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f82105ea740 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc5af9b4d0 0x55fc5b36e300 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f82105ea740 1 --2- 172.21.15.19:0/1516855128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc5af932f0 0x55fc5b36dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 >> 172.21.15.19:0/1516855128 conn(0x55fc5afb0300 msgr2=0x55fc5b35ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 shutdown_connections 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f82105ea740 1 -- 172.21.15.19:0/1516855128 wait complete. 2024-04-25T10:56:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f82105ea740 20 client.15720 trim_cache size 0 max 16384 2024-04-25T10:56:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:56:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:56:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:56:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f81f00afc50 msgr2=0x7f81f00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f81f00afc50 0x7f81f00b2160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0c38d0 tx=0x7f81fc0b0040 comp rx=0 tx=0).stop 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8204108f20 msgr2=0x7f820419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8204108f20 0x7f820419ef60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004770 tx=0x7f81fc004270 comp rx=0 tx=0).stop 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 shutdown_connections 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f81f00afc50 0x7f81f00b2160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8204108f20 0x7f820419ef60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8204108a80 0x7f820419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 --2- 172.21.15.19:0/3435109750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82041031d0 0x7f820419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.641+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 >> 172.21.15.19:0/3435109750 conn(0x7f82040781b0 msgr2=0x7f820406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.642+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 shutdown_connections 2024-04-25T10:56:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:08.642+0000 7f820973c640 1 -- 172.21.15.19:0/3435109750 wait complete. 2024-04-25T10:56:08.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:56:08.817 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 -- 172.21.15.19:0/2879926778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181081f0 msgr2=0x7f3018101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 --2- 172.21.15.19:0/2879926778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181081f0 0x7f3018101e80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3008009900 tx=0x7f300802a410 comp rx=0 tx=0).stop 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 -- 172.21.15.19:0/2879926778 shutdown_connections 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 --2- 172.21.15.19:0/2879926778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30181024f0 0x7f301810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 --2- 172.21.15.19:0/2879926778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181081f0 0x7f3018101e80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 --2- 172.21.15.19:0/2879926778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3018107d50 0x7f3018101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 -- 172.21.15.19:0/2879926778 >> 172.21.15.19:0/2879926778 conn(0x7f3018077440 msgr2=0x7f3018077880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 -- 172.21.15.19:0/2879926778 shutdown_connections 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 -- 172.21.15.19:0/2879926778 wait complete. 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 Processor -- start 2024-04-25T10:56:08.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.935+0000 7f301fdb7640 1 -- start start 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301fdb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181024f0 0x7f301819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301fdb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3018107d50 0x7f301819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301fdb7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30181081f0 0x7f301819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301fdb7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3018112c00 con 0x7f30181024f0 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301fdb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3018112a80 con 0x7f3018107d50 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301fdb7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3018112d80 con 0x7f30181081f0 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181024f0 0x7f301819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181024f0 0x7f301819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50134/0 (socket says 172.21.15.19:50134) 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 -- 172.21.15.19:0/3076787247 learned_addr learned my addr 172.21.15.19:0/3076787247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301e354640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30181081f0 0x7f301819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301d352640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3018107d50 0x7f301819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 -- 172.21.15.19:0/3076787247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30181081f0 msgr2=0x7f301819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30181081f0 0x7f301819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 -- 172.21.15.19:0/3076787247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3018107d50 msgr2=0x7f301819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3018107d50 0x7f301819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 -- 172.21.15.19:0/3076787247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301819f5d0 con 0x7f30181024f0 2024-04-25T10:56:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301e354640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30181081f0 0x7f301819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301db53640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181024f0 0x7f301819a770 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f301400c970 tx=0x7f301400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f300effd640 1 -- 172.21.15.19:0/3076787247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014007bb0 con 0x7f30181024f0 2024-04-25T10:56:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f300effd640 1 -- 172.21.15.19:0/3076787247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3014007d50 con 0x7f30181024f0 2024-04-25T10:56:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301819f8c0 con 0x7f30181024f0 2024-04-25T10:56:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.936+0000 7f300effd640 1 -- 172.21.15.19:0/3076787247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301400f450 con 0x7f30181024f0 2024-04-25T10:56:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.937+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181a7240 con 0x7f30181024f0 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.940+0000 7f300effd640 1 -- 172.21.15.19:0/3076787247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3014004f20 con 0x7f30181024f0 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.940+0000 7f3026c6c740 1 Processor -- start 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.940+0000 7f3026c6c740 1 -- start start 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3004d0 0x5604fc6c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f3026c6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5604fc62fd80 con 0x5604fc3086b0 2024-04-25T10:56:08.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f3026c6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604fc5e98b0 con 0x5604fc329390 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f3026c6c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5604fc62bd40 con 0x5604fc3004d0 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301db53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3004d0 0x5604fc6c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6dad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50150/0 (socket says 172.21.15.19:50150) 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301db53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3004d0 0x5604fc6c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:51576/0 (socket says 172.21.15.19:51576) 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 -- 172.21.15.19:0/618146698 learned_addr learned my addr 172.21.15.19:0/618146698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301e354640 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f300effd640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30040afce0 0x7f30040b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 -- 172.21.15.19:0/618146698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3004d0 msgr2=0x5604fc6c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3004d0 0x5604fc6c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301e354640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30040afce0 0x7f30040b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 -- 172.21.15.19:0/618146698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 msgr2=0x5604fc6c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f301d352640 1 -- 172.21.15.19:0/618146698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fc5baa40 con 0x5604fc3086b0 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.941+0000 7f300effd640 1 -- 172.21.15.19:0/3076787247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f30140def90 con 0x7f30181024f0 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.942+0000 7f301d352640 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6dad00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f3008032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a9efd730dc0592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.942+0000 7f3003fff640 1 -- 172.21.15.19:0/618146698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300803f070 con 0x5604fc3086b0 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.942+0000 7f301e354640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30040afce0 0x7f30040b21f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5604fc6c6c00 tx=0x7f3010004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.942+0000 7f3003fff640 1 -- 172.21.15.19:0/618146698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30080316a0 con 0x5604fc3086b0 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.942+0000 7f3003fff640 1 -- 172.21.15.19:0/618146698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008035070 con 0x5604fc3086b0 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 -- 172.21.15.19:0/618146698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 msgr2=0x5604fc6dad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6dad00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f3008032c30 comp rx=0 tx=0).stop 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 -- 172.21.15.19:0/618146698 shutdown_connections 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6dad00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3004d0 0x5604fc6c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 --2- 172.21.15.19:0/618146698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 -- 172.21.15.19:0/618146698 >> 172.21.15.19:0/618146698 conn(0x5604fc31d4e0 msgr2=0x5604fc6c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 -- 172.21.15.19:0/618146698 shutdown_connections 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 -- 172.21.15.19:0/618146698 wait complete. 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 -- start start 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.943+0000 7f3026c6c740 1 Processor -- start 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3004d0 0x5604fc6d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3086b0 0x5604fc6d8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f3026c6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5604fc6db390 con 0x5604fc3004d0 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f3026c6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604fc5b6d10 con 0x5604fc329390 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f3026c6c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5604fc6db510 con 0x5604fc3086b0 2024-04-25T10:56:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f3026c6c740 10 client.0 _ll_register_callbacks cb 0x5604fc5e3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3004d0 0x5604fc6d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301d352640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3086b0 0x5604fc6d8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3004d0 0x5604fc6d8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50162/0 (socket says 172.21.15.19:50162) 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 -- 172.21.15.19:0/323154592 learned_addr learned my addr 172.21.15.19:0/323154592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301e354640 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 -- 172.21.15.19:0/323154592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3086b0 msgr2=0x5604fc6d8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3086b0 0x5604fc6d8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 -- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 msgr2=0x5604fc6de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6de8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301db53640 1 -- 172.21.15.19:0/323154592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fc40b7d0 con 0x5604fc3004d0 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.944+0000 7f301d352640 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3086b0 0x5604fc6d8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:08.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f301db53640 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3004d0 0x5604fc6d8a10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f30140a31e0 tx=0x7f3014007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ce33befa1714ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f30017fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f30017fa640 1 -- 172.21.15.19:0/323154592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140dce60 con 0x5604fc3004d0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f30017fa640 1 -- 172.21.15.19:0/323154592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f30140e10f0 con 0x5604fc3004d0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30080316a0 con 0x5604fc3004d0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f30017fa640 1 -- 172.21.15.19:0/323154592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f301401d040 con 0x5604fc3004d0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 10 client.15732 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5604fc3413f0 con 0x5604fc3004d0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f30017fa640 1 -- 172.21.15.19:0/323154592 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30080316a0 con 0x5604fc3004d0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f30017fa640 1 -- 172.21.15.19:0/323154592 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f30140ab120 con 0x5604fc3004d0 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 10 client.15732 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 20 client.15732 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2e7f60 0x5604fc6eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 4 client.15732 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.945+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604fc63d990 con 0x5604fc2e7f60 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.946+0000 7f301d352640 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2e7f60 0x5604fc6eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.946+0000 7f30017fa640 10 client.15732 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.946+0000 7f301d352640 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2e7f60 0x5604fc6eaac0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24fbac6a79faff85 server_cookie=12db8a25cb27b5fb in_seq=0 out_seq=0 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f30017fa640 1 -- 172.21.15.19:0/323154592 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5604fc63d990 con 0x5604fc2e7f60 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f30017fa640 10 client.15732 handle_command_reply: tid=0 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 client.15732 shutdown 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3004d0 msgr2=0x5604fc6d8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3004d0 0x5604fc6d8a10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f30140a31e0 tx=0x7f3014007f80 comp rx=0 tx=0).stop 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 shutdown_connections 2024-04-25T10:56:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2e7f60 0x5604fc6eaac0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc329390 0x5604fc6de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f30017fa640 0 client.15732 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc3086b0 0x5604fc6d8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 --2- 172.21.15.19:0/323154592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3004d0 0x5604fc6d8a10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 >> 172.21.15.19:0/323154592 conn(0x5604fc31d4e0 msgr2=0x5604fc6cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 shutdown_connections 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 1 -- 172.21.15.19:0/323154592 wait complete. 2024-04-25T10:56:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.947+0000 7f3026c6c740 20 client.15732 trim_cache size 0 max 16384 2024-04-25T10:56:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 Processor -- start 2024-04-25T10:56:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 -- start start 2024-04-25T10:56:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6d7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6de810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6ded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30181ab720 con 0x5604fc3086b0 2024-04-25T10:56:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30181adf80 con 0x5604fc3004d0 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f3026c6c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30181abd70 con 0x5604fc329390 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f301d352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6de810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f301d352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6de810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50170/0 (socket says 172.21.15.19:50170) 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f301d352640 1 -- 172.21.15.19:0/949169621 learned_addr learned my addr 172.21.15.19:0/949169621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.967+0000 7f301e354640 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6ded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301db53640 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6d7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301e354640 1 -- 172.21.15.19:0/949169621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 msgr2=0x5604fc6d7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301e354640 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6d7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301e354640 1 -- 172.21.15.19:0/949169621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 msgr2=0x5604fc6de810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301e354640 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6de810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301e354640 1 -- 172.21.15.19:0/949169621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fc3010f0 con 0x5604fc329390 2024-04-25T10:56:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301d352640 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6de810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301db53640 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f301e354640 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6ded50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30100049d0 tx=0x7f3010015050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d8c13d878ca710 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f30037fe640 1 -- 172.21.15.19:0/949169621 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301001e500 con 0x5604fc329390 2024-04-25T10:56:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f30037fe640 1 -- 172.21.15.19:0/949169621 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301001eb50 con 0x5604fc329390 2024-04-25T10:56:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.968+0000 7f30037fe640 1 -- 172.21.15.19:0/949169621 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010010af0 con 0x5604fc329390 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 -- 172.21.15.19:0/949169621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 msgr2=0x5604fc6ded50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6ded50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30100049d0 tx=0x7f3010015050 comp rx=0 tx=0).stop 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 -- 172.21.15.19:0/949169621 shutdown_connections 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6ded50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6de810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 --2- 172.21.15.19:0/949169621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 -- 172.21.15.19:0/949169621 >> 172.21.15.19:0/949169621 conn(0x5604fc31d4e0 msgr2=0x5604fc6cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 -- 172.21.15.19:0/949169621 shutdown_connections 2024-04-25T10:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 -- 172.21.15.19:0/949169621 wait complete. 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.969+0000 7f3026c6c740 1 -- start start 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 1 Processor -- start 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6e0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30181ac440 con 0x5604fc3086b0 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30181ad8b0 con 0x5604fc3004d0 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30181acb10 con 0x5604fc329390 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f3026c6c740 10 client.0 _ll_register_callbacks cb 0x7f30140e27d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301d352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301d352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6e0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50176/0 (socket says 172.21.15.19:50176) 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301d352640 1 -- 172.21.15.19:0/44711786 learned_addr learned my addr 172.21.15.19:0/44711786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301e354640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301db53640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301d352640 1 -- 172.21.15.19:0/44711786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 msgr2=0x5604fc6e07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301d352640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6e07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301d352640 1 -- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 msgr2=0x5604fc6dfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.970+0000 7f301d352640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6dfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f301d352640 1 -- 172.21.15.19:0/44711786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fc5f04b0 con 0x5604fc3086b0 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f301e354640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6e07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f301d352640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6e0280 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3008009900 tx=0x7f300802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ba1328c999eb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f30017fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f30017fa640 1 -- 172.21.15.19:0/44711786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008056070 con 0x5604fc3086b0 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f30017fa640 1 -- 172.21.15.19:0/44711786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3008004240 con 0x5604fc3086b0 2024-04-25T10:56:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301001eb50 con 0x5604fc3086b0 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f30017fa640 1 -- 172.21.15.19:0/44711786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3008062050 con 0x5604fc3086b0 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f3026c6c740 10 client.15738 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5604fc752100 con 0x5604fc3086b0 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f30017fa640 1 -- 172.21.15.19:0/44711786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301001eb50 con 0x5604fc3086b0 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f30017fa640 1 -- 172.21.15.19:0/44711786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f30080037b0 con 0x5604fc3086b0 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f3026c6c740 10 client.15738 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f3026c6c740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.971+0000 7f3026c6c740 20 client.15738 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f3026c6c740 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2bd3a0 0x5604fc754480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f3026c6c740 4 client.15738 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604fc637800 con 0x5604fc2bd3a0 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f301db53640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2bd3a0 0x5604fc754480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f30017fa640 10 client.15738 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f301db53640 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2bd3a0 0x5604fc754480 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75ab33f49f3c610 server_cookie=bcaf2f902ffd8001 in_seq=0 out_seq=0 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f30017fa640 1 -- 172.21.15.19:0/44711786 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604fc637800 con 0x5604fc2bd3a0 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f30017fa640 10 client.15738 handle_command_reply: tid=0 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.972+0000 7f3026c6c740 1 client.15738 shutdown 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 msgr2=0x5604fc6e0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6e0280 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3008009900 tx=0x7f300802a410 comp rx=0 tx=0).stop 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 shutdown_connections 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5604fc2bd3a0 0x5604fc754480 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604fc329390 0x5604fc6e07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604fc3086b0 0x5604fc6e0280 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f30017fa640 0 client.15738 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 --2- 172.21.15.19:0/44711786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604fc3004d0 0x5604fc6dfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 >> 172.21.15.19:0/44711786 conn(0x5604fc31d4e0 msgr2=0x5604fc6d9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 shutdown_connections 2024-04-25T10:56:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 1 -- 172.21.15.19:0/44711786 wait complete. 2024-04-25T10:56:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f3026c6c740 20 client.15738 trim_cache size 0 max 16384 2024-04-25T10:56:08.981 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:56:08.981 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:56:08.981 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:56:08.981 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:56:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30040afce0 msgr2=0x7f30040b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30040afce0 0x7f30040b21f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5604fc6c6c00 tx=0x7f3010004ab0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181024f0 msgr2=0x7f301819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181024f0 0x7f301819a770 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f301400c970 tx=0x7f301400ce30 comp rx=0 tx=0).stop 2024-04-25T10:56:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 shutdown_connections 2024-04-25T10:56:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30040afce0 0x7f30040b21f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30181081f0 0x7f301819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3018107d50 0x7f301819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 --2- 172.21.15.19:0/3076787247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30181024f0 0x7f301819a770 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.973+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 >> 172.21.15.19:0/3076787247 conn(0x7f3018077440 msgr2=0x7f3018105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.974+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 shutdown_connections 2024-04-25T10:56:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:08.974+0000 7f301fdb7640 1 -- 172.21.15.19:0/3076787247 wait complete. 2024-04-25T10:56:08.986 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:56:08.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 -- 172.21.15.19:0/1840580293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b410e040 msgr2=0x7f67b4110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/1840580293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b410e040 0x7f67b4110530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009ca0 tx=0x7f67ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 -- 172.21.15.19:0/1840580293 shutdown_connections 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/1840580293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b410e040 0x7f67b4110530 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/1840580293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b410b650 0x7f67b410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/1840580293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67b4104bf0 0x7f67b4105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 -- 172.21.15.19:0/1840580293 >> 172.21.15.19:0/1840580293 conn(0x7f67b4078110 msgr2=0x7f67b4078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 -- 172.21.15.19:0/1840580293 shutdown_connections 2024-04-25T10:56:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.103+0000 7f67ba2f9640 1 -- 172.21.15.19:0/1840580293 wait complete. 2024-04-25T10:56:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 Processor -- start 2024-04-25T10:56:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 -- start start 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b4104bf0 0x7f67b419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b410b650 0x7f67b419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67b410e040 0x7f67b419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4113ca0 con 0x7f67b4104bf0 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4113b20 con 0x7f67b410e040 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67ba2f9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4113e20 con 0x7f67b410b650 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b4104bf0 0x7f67b419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b4104bf0 0x7f67b419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50192/0 (socket says 172.21.15.19:50192) 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 -- 172.21.15.19:0/110118015 learned_addr learned my addr 172.21.15.19:0/110118015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b37fe640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b410b650 0x7f67b419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b8896640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67b410e040 0x7f67b419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 -- 172.21.15.19:0/110118015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b410b650 msgr2=0x7f67b419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b410b650 0x7f67b419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 -- 172.21.15.19:0/110118015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67b410e040 msgr2=0x7f67b419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67b410e040 0x7f67b419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.104+0000 7f67b3fff640 1 -- 172.21.15.19:0/110118015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b419f4a0 con 0x7f67b4104bf0 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.105+0000 7f67b8896640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67b410e040 0x7f67b419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.105+0000 7f67b3fff640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b4104bf0 0x7f67b419a640 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f67a400e9f0 tx=0x7f67a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.105+0000 7f67b17fa640 1 -- 172.21.15.19:0/110118015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400cd80 con 0x7f67b4104bf0 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.105+0000 7f67b17fa640 1 -- 172.21.15.19:0/110118015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a40108e0 con 0x7f67b4104bf0 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.105+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b419f790 con 0x7f67b4104bf0 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.105+0000 7f67b17fa640 1 -- 172.21.15.19:0/110118015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4018980 con 0x7f67b4104bf0 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.105+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41a7110 con 0x7f67b4104bf0 2024-04-25T10:56:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.106+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b406a770 con 0x7f67b4104bf0 2024-04-25T10:56:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.108+0000 7f67b17fa640 1 -- 172.21.15.19:0/110118015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f67a4010430 con 0x7f67b4104bf0 2024-04-25T10:56:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.108+0000 7f67b17fa640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f679c0afce0 0x7f679c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.108+0000 7f67b17fa640 1 -- 172.21.15.19:0/110118015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f67a40df830 con 0x7f67b4104bf0 2024-04-25T10:56:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.108+0000 7f67b37fe640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f679c0afce0 0x7f679c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.109+0000 7f67b37fe640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f679c0afce0 0x7f679c0b21f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67b410e870 tx=0x7f67a8005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.112+0000 7f67b17fa640 1 -- 172.21.15.19:0/110118015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67a40a8100 con 0x7f67b4104bf0 2024-04-25T10:56:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.340+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67b406afa0 con 0x7f67b4104bf0 2024-04-25T10:56:09.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.341+0000 7f67b17fa640 1 -- 172.21.15.19:0/110118015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f67a40b0180 con 0x7f67b4104bf0 2024-04-25T10:56:09.342 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:56:09.342 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:56:09.342 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f679c0afce0 msgr2=0x7f679c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f679c0afce0 0x7f679c0b21f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67b410e870 tx=0x7f67a8005f90 comp rx=0 tx=0).stop 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b4104bf0 msgr2=0x7f67b419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b4104bf0 0x7f67b419a640 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f67a400e9f0 tx=0x7f67a400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 shutdown_connections 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f679c0afce0 0x7f679c0b21f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67b410e040 0x7f67b419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b410b650 0x7f67b419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 --2- 172.21.15.19:0/110118015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b4104bf0 0x7f67b419a640 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 >> 172.21.15.19:0/110118015 conn(0x7f67b4078110 msgr2=0x7f67b4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 shutdown_connections 2024-04-25T10:56:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.343+0000 7f67ba2f9640 1 -- 172.21.15.19:0/110118015 wait complete. 2024-04-25T10:56:09.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:56:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:09 smithi019 ceph-mon[29209]: pgmap v52: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:09.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:09 smithi169 ceph-mon[33049]: pgmap v52: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.512+0000 7f0db8ef2640 1 -- 172.21.15.19:0/2893812863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 msgr2=0x7f0db410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.512+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/2893812863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db410f7a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ca0 tx=0x7f0dac02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.512+0000 7f0db8ef2640 1 -- 172.21.15.19:0/2893812863 shutdown_connections 2024-04-25T10:56:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.512+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/2893812863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db410f7a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.512+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/2893812863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db4102af0 0x7f0db4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.512+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/2893812863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db4108a20 0x7f0db41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.513+0000 7f0db8ef2640 1 -- 172.21.15.19:0/2893812863 >> 172.21.15.19:0/2893812863 conn(0x7f0db4078100 msgr2=0x7f0db4078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.513+0000 7f0db8ef2640 1 -- 172.21.15.19:0/2893812863 shutdown_connections 2024-04-25T10:56:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.513+0000 7f0db8ef2640 1 -- 172.21.15.19:0/2893812863 wait complete. 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.513+0000 7f0db8ef2640 1 Processor -- start 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.513+0000 7f0db8ef2640 1 -- start start 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db8ef2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db4102af0 0x7f0db419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db8ef2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db4108a20 0x7f0db419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db8ef2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db8ef2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0db4112c00 con 0x7f0db410d380 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db8ef2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0db4112a80 con 0x7f0db4108a20 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db8ef2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0db4112d80 con 0x7f0db4102af0 2024-04-25T10:56:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50198/0 (socket says 172.21.15.19:50198) 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db4102af0 0x7f0db419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 -- 172.21.15.19:0/1448271507 learned_addr learned my addr 172.21.15.19:0/1448271507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db4108a20 0x7f0db419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 -- 172.21.15.19:0/1448271507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db4102af0 msgr2=0x7f0db419b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db4102af0 0x7f0db419b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 -- 172.21.15.19:0/1448271507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db4108a20 msgr2=0x7f0db419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db4108a20 0x7f0db419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 -- 172.21.15.19:0/1448271507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db41a6850 con 0x7f0db410d380 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.514+0000 7f0db2d9d640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db419bfc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0098d0 tx=0x7f0dac0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.515+0000 7f0da37fe640 1 -- 172.21.15.19:0/1448271507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac003b70 con 0x7f0db410d380 2024-04-25T10:56:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.515+0000 7f0da37fe640 1 -- 172.21.15.19:0/1448271507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dac003d10 con 0x7f0db410d380 2024-04-25T10:56:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.515+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db41a6ae0 con 0x7f0db410d380 2024-04-25T10:56:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.515+0000 7f0da37fe640 1 -- 172.21.15.19:0/1448271507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac02fa20 con 0x7f0db410d380 2024-04-25T10:56:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.515+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db41a7080 con 0x7f0db410d380 2024-04-25T10:56:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.517+0000 7f0dbfda6740 1 Processor -- start 2024-04-25T10:56:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0da37fe640 1 -- 172.21.15.19:0/1448271507 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0dac02fc40 con 0x7f0db410d380 2024-04-25T10:56:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0dbfda6740 1 -- start start 2024-04-25T10:56:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5f5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f5bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5571d5ec0d80 con 0x5571d5b996b0 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5571d5e7aa20 con 0x5571d5bba390 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5571d5ebcd40 con 0x5571d5b914d0 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0db259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f5bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0db259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f5bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:51648/0 (socket says 172.21.15.19:51648) 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0db1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0db259c640 1 -- 172.21.15.19:0/3784921806 learned_addr learned my addr 172.21.15.19:0/3784921806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.518+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5f5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0da37fe640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f0d9c0afce0 0x7f0d9c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db2d9d640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f0d9c0afce0 0x7f0d9c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db259c640 1 -- 172.21.15.19:0/3784921806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 msgr2=0x5571d5f5b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db259c640 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5f5b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db259c640 1 -- 172.21.15.19:0/3784921806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 msgr2=0x5571d5f686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db259c640 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db259c640 1 -- 172.21.15.19:0/3784921806 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d5e85c30 con 0x5571d5b914d0 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0da37fe640 1 -- 172.21.15.19:0/1448271507 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f0dac100370 con 0x7f0db410d380 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db259c640 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f5bd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0da400bc30 tx=0x7f0da40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50513094c38d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0db2d9d640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f0d9c0afce0 0x7f0d9c0b21f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0c5860 tx=0x7f0dac095e50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0da0ff9640 1 -- 172.21.15.19:0/3784921806 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da401c480 con 0x5571d5b914d0 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0da0ff9640 1 -- 172.21.15.19:0/3784921806 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da401cad0 con 0x5571d5b914d0 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.519+0000 7f0da0ff9640 1 -- 172.21.15.19:0/3784921806 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4011a20 con 0x5571d5b914d0 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 -- 172.21.15.19:0/3784921806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 msgr2=0x5571d5f5bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f5bd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0da400bc30 tx=0x7f0da40107a0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 -- 172.21.15.19:0/3784921806 shutdown_connections 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f5bd50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/3784921806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5f5b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 -- 172.21.15.19:0/3784921806 >> 172.21.15.19:0/3784921806 conn(0x5571d5bae4e0 msgr2=0x5571d5f57fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 -- 172.21.15.19:0/3784921806 shutdown_connections 2024-04-25T10:56:09.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.520+0000 7f0dbfda6740 1 -- 172.21.15.19:0/3784921806 wait complete. 2024-04-25T10:56:09.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 -- start start 2024-04-25T10:56:09.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 Processor -- start 2024-04-25T10:56:09.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b914d0 0x5571d5f66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b996b0 0x5571d5f667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5f72250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5571d5f68d90 con 0x5571d5b914d0 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5571d5e47dc0 con 0x5571d5bba390 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5571d5f68f10 con 0x5571d5b996b0 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0dbfda6740 10 client.0 _ll_register_callbacks cb 0x5571d5e63600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0db1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b914d0 0x5571d5f66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0db1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b914d0 0x5571d5f66260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50212/0 (socket says 172.21.15.19:50212) 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0db1d9b640 1 -- 172.21.15.19:0/1370895979 learned_addr learned my addr 172.21.15.19:0/1370895979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.521+0000 7f0db259c640 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b996b0 0x5571d5f667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0db1d9b640 1 -- 172.21.15.19:0/1370895979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b996b0 msgr2=0x5571d5f667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b996b0 0x5571d5f667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0db1d9b640 1 -- 172.21.15.19:0/1370895979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 msgr2=0x5571d5f72250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5f72250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0db1d9b640 1 -- 172.21.15.19:0/1370895979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d5bd23f0 con 0x5571d5b914d0 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b914d0 0x5571d5f66260 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0db406c4d0 tx=0x7f0da800e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7819215b952e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0d99ffb640 1 -- 172.21.15.19:0/1370895979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8019040 con 0x5571d5b914d0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0d99ffb640 1 -- 172.21.15.19:0/1370895979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f0da8004050 con 0x5571d5b914d0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da401cad0 con 0x5571d5b914d0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0d99ffb640 1 -- 172.21.15.19:0/1370895979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0da8004250 con 0x5571d5b914d0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0dbfda6740 10 client.15756 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.522+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5571d5c9c7d0 con 0x5571d5b914d0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0d99ffb640 1 -- 172.21.15.19:0/1370895979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da401cad0 con 0x5571d5b914d0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0d99ffb640 1 -- 172.21.15.19:0/1370895979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0da801f410 con 0x5571d5b914d0 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0dbfda6740 10 client.15756 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0dbfda6740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:09.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0dbfda6740 20 client.15756 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b78f60 0x5571d5f78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0dbfda6740 4 client.15756 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571d5ece990 con 0x5571d5b78f60 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.523+0000 7f0db259c640 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b78f60 0x5571d5f78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.524+0000 7f0db259c640 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b78f60 0x5571d5f78760 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d128982e3bd7dd server_cookie=3dec2bc140a96d40 in_seq=0 out_seq=0 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.524+0000 7f0d99ffb640 10 client.15756 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.524+0000 7f0d99ffb640 1 -- 172.21.15.19:0/1370895979 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5571d5ece990 con 0x5571d5b78f60 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.524+0000 7f0d99ffb640 10 client.15756 handle_command_reply: tid=0 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.524+0000 7f0dbfda6740 1 client.15756 shutdown 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b914d0 msgr2=0x5571d5f66260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b914d0 0x5571d5f66260 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0db406c4d0 tx=0x7f0da800e780 comp rx=0 tx=0).stop 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 shutdown_connections 2024-04-25T10:56:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b78f60 0x5571d5f78760 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5f72250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b996b0 0x5571d5f667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/1370895979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b914d0 0x5571d5f66260 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 >> 172.21.15.19:0/1370895979 conn(0x5571d5bae4e0 msgr2=0x5571d5f5d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0d99ffb640 0 client.15756 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 shutdown_connections 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 1 -- 172.21.15.19:0/1370895979 wait complete. 2024-04-25T10:56:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.525+0000 7f0dbfda6740 20 client.15756 trim_cache size 0 max 16384 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.538+0000 7f0dbfda6740 1 Processor -- start 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.538+0000 7f0dbfda6740 1 -- start start 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f71e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5b996b0 0x5571d5f56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5bba390 0x5571d5f57260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41ad990 con 0x5571d5bba390 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41b0210 con 0x5571d5b996b0 2024-04-25T10:56:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41ae000 con 0x5571d5b914d0 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f71e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5bba390 0x5571d5f57260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5bba390 0x5571d5f57260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50228/0 (socket says 172.21.15.19:50228) 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 -- 172.21.15.19:0/708016762 learned_addr learned my addr 172.21.15.19:0/708016762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db259c640 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5b996b0 0x5571d5f56d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 -- 172.21.15.19:0/708016762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 msgr2=0x5571d5f71e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f71e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 -- 172.21.15.19:0/708016762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5b996b0 msgr2=0x5571d5f56d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5b996b0 0x5571d5f56d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 -- 172.21.15.19:0/708016762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d5f5b5a0 con 0x5571d5bba390 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.539+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5bba390 0x5571d5f57260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0da801f7b0 tx=0x7f0da8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2698e837673304c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0d9bfff640 1 -- 172.21.15.19:0/708016762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da802c070 con 0x5571d5bba390 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0d9bfff640 1 -- 172.21.15.19:0/708016762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da802d040 con 0x5571d5bba390 2024-04-25T10:56:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0d9bfff640 1 -- 172.21.15.19:0/708016762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da801fde0 con 0x5571d5bba390 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 -- 172.21.15.19:0/708016762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5bba390 msgr2=0x5571d5f57260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5bba390 0x5571d5f57260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0da801f7b0 tx=0x7f0da8009780 comp rx=0 tx=0).stop 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 -- 172.21.15.19:0/708016762 shutdown_connections 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5bba390 0x5571d5f57260 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5b996b0 0x5571d5f56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/708016762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f71e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 -- 172.21.15.19:0/708016762 >> 172.21.15.19:0/708016762 conn(0x5571d5bae4e0 msgr2=0x5571d5f56610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 -- 172.21.15.19:0/708016762 shutdown_connections 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.540+0000 7f0dbfda6740 1 -- 172.21.15.19:0/708016762 wait complete. 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 -- start start 2024-04-25T10:56:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 Processor -- start 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f69150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f69690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5fe2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41ae6d0 con 0x5571d5b996b0 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41afb40 con 0x5571d5bba390 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0db41aeda0 con 0x5571d5b914d0 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0dbfda6740 10 client.0 _ll_register_callbacks cb 0x7f0da80104c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f69690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f69690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50234/0 (socket says 172.21.15.19:50234) 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db259c640 1 -- 172.21.15.19:0/79161034 learned_addr learned my addr 172.21.15.19:0/79161034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f69150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db259c640 1 -- 172.21.15.19:0/79161034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 msgr2=0x5571d5f69150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db259c640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f69150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5fe2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.541+0000 7f0db259c640 1 -- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 msgr2=0x5571d5fe2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0db259c640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5fe2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0db259c640 1 -- 172.21.15.19:0/79161034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d5f69f60 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5fe2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0db259c640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f69690 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0da4011010 tx=0x7f0da400b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad73494221d2b22f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0d99ffb640 1 -- 172.21.15.19:0/79161034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4005040 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0d99ffb640 1 -- 172.21.15.19:0/79161034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f0da40051e0 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da802d040 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0d99ffb640 1 -- 172.21.15.19:0/79161034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0da403a050 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 10 client.15762 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5571d5fe4490 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0d99ffb640 1 -- 172.21.15.19:0/79161034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da802d040 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0d99ffb640 1 -- 172.21.15.19:0/79161034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0da4004680 con 0x5571d5b996b0 2024-04-25T10:56:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 10 client.15762 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 20 client.15762 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b4e3a0 0x5571d5fe6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 4 client.15762 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.542+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571d5ec8800 con 0x5571d5b4e3a0 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b4e3a0 0x5571d5fe6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0d99ffb640 10 client.15762 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0db1d9b640 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b4e3a0 0x5571d5fe6910 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129386955c83bc6c server_cookie=516b41f2ebc1c35b in_seq=0 out_seq=0 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0d99ffb640 1 -- 172.21.15.19:0/79161034 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571d5ec8800 con 0x5571d5b4e3a0 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0d99ffb640 10 client.15762 handle_command_reply: tid=0 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0dbfda6740 1 client.15762 shutdown 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 msgr2=0x5571d5f69690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.543+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f69690 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0da4011010 tx=0x7f0da400b570 comp rx=0 tx=0).stop 2024-04-25T10:56:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 shutdown_connections 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5571d5b4e3a0 0x5571d5fe6910 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571d5bba390 0x5571d5fe2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571d5b996b0 0x5571d5f69690 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0d99ffb640 0 client.15762 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 --2- 172.21.15.19:0/79161034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5571d5b914d0 0x5571d5f69150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 >> 172.21.15.19:0/79161034 conn(0x5571d5bae4e0 msgr2=0x5571d5f67580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 shutdown_connections 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 1 -- 172.21.15.19:0/79161034 wait complete. 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0dbfda6740 20 client.15762 trim_cache size 0 max 16384 2024-04-25T10:56:09.550 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:56:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f0d9c0afce0 msgr2=0x7f0d9c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f0d9c0afce0 0x7f0d9c0b21f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0c5860 tx=0x7f0dac095e50 comp rx=0 tx=0).stop 2024-04-25T10:56:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 msgr2=0x7f0db419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db419bfc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0098d0 tx=0x7f0dac0317b0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 shutdown_connections 2024-04-25T10:56:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f0d9c0afce0 0x7f0d9c0b21f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db410d380 0x7f0db419bfc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db4108a20 0x7f0db419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 --2- 172.21.15.19:0/1448271507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db4102af0 0x7f0db419b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 >> 172.21.15.19:0/1448271507 conn(0x7f0db4078100 msgr2=0x7f0db41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 shutdown_connections 2024-04-25T10:56:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:09.544+0000 7f0db8ef2640 1 -- 172.21.15.19:0/1448271507 wait complete. 2024-04-25T10:56:09.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:09 smithi112 ceph-mon[32799]: pgmap v52: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:10.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:10 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/110118015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:56:10.557 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:56:10.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:56:10.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:56:10.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:10 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/110118015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:56:10.580 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:10 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/110118015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:56:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 -- 172.21.15.19:0/1210776152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f664010e060 msgr2=0x7f6640110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 --2- 172.21.15.19:0/1210776152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f664010e060 0x7f6640110550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6638009ca0 tx=0x7f663802a9e0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 -- 172.21.15.19:0/1210776152 shutdown_connections 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 --2- 172.21.15.19:0/1210776152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f664010e060 0x7f6640110550 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 --2- 172.21.15.19:0/1210776152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f664010b6b0 0x7f664010dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 --2- 172.21.15.19:0/1210776152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6640104c50 0x7f6640105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 -- 172.21.15.19:0/1210776152 >> 172.21.15.19:0/1210776152 conn(0x7f6640078100 msgr2=0x7f6640078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 -- 172.21.15.19:0/1210776152 shutdown_connections 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.690+0000 7f664674d640 1 -- 172.21.15.19:0/1210776152 wait complete. 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.691+0000 7f664674d640 1 Processor -- start 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.691+0000 7f664674d640 1 -- start start 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.691+0000 7f664674d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6640104c50 0x7f664019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.691+0000 7f664674d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664010b6b0 0x7f664019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.691+0000 7f664674d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f664010e060 0x7f664019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f664674d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6640113d10 con 0x7f6640104c50 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f664674d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6640113b90 con 0x7f664010b6b0 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f664674d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6640113e90 con 0x7f664010e060 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6640104c50 0x7f664019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6640104c50 0x7f664019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50244/0 (socket says 172.21.15.19:50244) 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 -- 172.21.15.19:0/4072071434 learned_addr learned my addr 172.21.15.19:0/4072071434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f6644cea640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f664010e060 0x7f664019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663f7fe640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664010b6b0 0x7f664019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 -- 172.21.15.19:0/4072071434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f664010e060 msgr2=0x7f664019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f664010e060 0x7f664019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 -- 172.21.15.19:0/4072071434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664010b6b0 msgr2=0x7f664019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664010b6b0 0x7f664019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 -- 172.21.15.19:0/4072071434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664019f570 con 0x7f6640104c50 2024-04-25T10:56:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663f7fe640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664010b6b0 0x7f664019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663ffff640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6640104c50 0x7f664019a710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f663000e9f0 tx=0x7f663000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663d7fa640 1 -- 172.21.15.19:0/4072071434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663000cd80 con 0x7f6640104c50 2024-04-25T10:56:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664019f860 con 0x7f6640104c50 2024-04-25T10:56:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663d7fa640 1 -- 172.21.15.19:0/4072071434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66300108e0 con 0x7f6640104c50 2024-04-25T10:56:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f663d7fa640 1 -- 172.21.15.19:0/4072071434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630018980 con 0x7f6640104c50 2024-04-25T10:56:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.692+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66401a71e0 con 0x7f6640104c50 2024-04-25T10:56:10.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f663d7fa640 1 -- 172.21.15.19:0/4072071434 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6630010430 con 0x7f6640104c50 2024-04-25T10:56:10.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 Processor -- start 2024-04-25T10:56:10.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 -- start start 2024-04-25T10:56:10.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7138410 0x5652e74d8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e710f550 0x5652e74d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5652e743eb00 con 0x5652e710f550 2024-04-25T10:56:10.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5652e73da920 con 0x5652e7117730 2024-04-25T10:56:10.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.695+0000 7f664d601740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5652e743aac0 con 0x5652e7138410 2024-04-25T10:56:10.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e710f550 0x5652e74d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e710f550 0x5652e74d8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50260/0 (socket says 172.21.15.19:50260) 2024-04-25T10:56:10.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 -- 172.21.15.19:0/4198968773 learned_addr learned my addr 172.21.15.19:0/4198968773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f6644cea640 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7138410 0x5652e74d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663d7fa640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f66280afc60 0x7f66280b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663f7fe640 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f6644cea640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f66280afc60 0x7f66280b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663d7fa640 1 -- 172.21.15.19:0/4072071434 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f66300df3d0 con 0x7f6640104c50 2024-04-25T10:56:10.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 -- 172.21.15.19:0/4198968773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7138410 msgr2=0x5652e74d8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7138410 0x5652e74d8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 -- 172.21.15.19:0/4198968773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 msgr2=0x5652e74d90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74d90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 -- 172.21.15.19:0/4198968773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e72187e0 con 0x5652e710f550 2024-04-25T10:56:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f663ffff640 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e710f550 0x5652e74d8b60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f66300dcaf0 tx=0x7f663000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13535851bc49d32e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f6644cea640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f66280afc60 0x7f66280b2170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66380094e0 tx=0x7f663802a9e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.696+0000 7f66267fc640 1 -- 172.21.15.19:0/4198968773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630010720 con 0x5652e710f550 2024-04-25T10:56:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f66267fc640 1 -- 172.21.15.19:0/4198968773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66300a70b0 con 0x5652e710f550 2024-04-25T10:56:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f66267fc640 1 -- 172.21.15.19:0/4198968773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66300b0400 con 0x5652e710f550 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 -- 172.21.15.19:0/4198968773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e710f550 msgr2=0x5652e74d8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e710f550 0x5652e74d8b60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f66300dcaf0 tx=0x7f663000cf20 comp rx=0 tx=0).stop 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 -- 172.21.15.19:0/4198968773 shutdown_connections 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74d90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e710f550 0x5652e74d8b60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 --2- 172.21.15.19:0/4198968773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7138410 0x5652e74d8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 -- 172.21.15.19:0/4198968773 >> 172.21.15.19:0/4198968773 conn(0x5652e712c560 msgr2=0x5652e74d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 -- 172.21.15.19:0/4198968773 shutdown_connections 2024-04-25T10:56:10.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.697+0000 7f664d601740 1 -- 172.21.15.19:0/4198968773 wait complete. 2024-04-25T10:56:10.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.698+0000 7f664d601740 1 -- start start 2024-04-25T10:56:10.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.698+0000 7f664d601740 1 Processor -- start 2024-04-25T10:56:10.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e710f550 0x5652e74e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7117730 0x5652e74e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5652e74e58b0 con 0x5652e7138410 2024-04-25T10:56:10.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5652e73d85d0 con 0x5652e710f550 2024-04-25T10:56:10.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5652e74e5a30 con 0x5652e7117730 2024-04-25T10:56:10.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 10 client.0 _ll_register_callbacks cb 0x5652e73c6080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:10.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7117730 0x5652e74e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74eed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50274/0 (socket says 172.21.15.19:50274) 2024-04-25T10:56:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 -- 172.21.15.19:0/591564108 learned_addr learned my addr 172.21.15.19:0/591564108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f663ffff640 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e710f550 0x5652e74e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 -- 172.21.15.19:0/591564108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7117730 msgr2=0x5652e74e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7117730 0x5652e74e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 -- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e710f550 msgr2=0x5652e74e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e710f550 0x5652e74e2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 -- 172.21.15.19:0/591564108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e7278f70 con 0x5652e7138410 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f663f7fe640 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7117730 0x5652e74e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f6644cea640 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74eed60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6638009ca0 tx=0x7f66380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba2421da4bdebad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f661f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f661f7fe640 1 -- 172.21.15.19:0/591564108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638033040 con 0x5652e7138410 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f661f7fe640 1 -- 172.21.15.19:0/591564108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6638038650 con 0x5652e7138410 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.699+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66300a70b0 con 0x5652e7138410 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f661f7fe640 1 -- 172.21.15.19:0/591564108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f663802fda0 con 0x5652e7138410 2024-04-25T10:56:10.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 10 client.15774 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5652e7110170 con 0x5652e7138410 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f661f7fe640 1 -- 172.21.15.19:0/591564108 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66300a70b0 con 0x5652e7138410 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f661f7fe640 1 -- 172.21.15.19:0/591564108 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f66380538b0 con 0x5652e7138410 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 10 client.15774 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 20 client.15774 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70f71f0 0x5652e74f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 4 client.15774 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.700+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652e744c710 con 0x5652e70f71f0 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.701+0000 7f663ffff640 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70f71f0 0x5652e74f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.701+0000 7f663ffff640 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70f71f0 0x5652e74f4ed0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7d85b52bb4f84e server_cookie=d7e3cb68f8f2f54a in_seq=0 out_seq=0 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.701+0000 7f661f7fe640 10 client.15774 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f661f7fe640 1 -- 172.21.15.19:0/591564108 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652e744c710 con 0x5652e70f71f0 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f661f7fe640 10 client.15774 handle_command_reply: tid=0 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 client.15774 shutdown 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 msgr2=0x5652e74eed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74eed60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6638009ca0 tx=0x7f66380095c0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 shutdown_connections 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70f71f0 0x5652e74f4ed0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74eed60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e7117730 0x5652e74e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 --2- 172.21.15.19:0/591564108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e710f550 0x5652e74e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f661f7fe640 0 client.15774 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 >> 172.21.15.19:0/591564108 conn(0x5652e712c560 msgr2=0x5652e74da340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 shutdown_connections 2024-04-25T10:56:10.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 1 -- 172.21.15.19:0/591564108 wait complete. 2024-04-25T10:56:10.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.702+0000 7f664d601740 20 client.15774 trim_cache size 0 max 16384 2024-04-25T10:56:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.725+0000 7f664d601740 1 Processor -- start 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f664d601740 1 -- start start 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f664d601740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f664d601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f664d601740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74fabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f664d601740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66401ab840 con 0x5652e7138410 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f664d601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66401ae0c0 con 0x5652e7117730 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f664d601740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66401abeb0 con 0x5652e710f550 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74fabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74fabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50288/0 (socket says 172.21.15.19:50288) 2024-04-25T10:56:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 -- 172.21.15.19:0/3583617879 learned_addr learned my addr 172.21.15.19:0/3583617879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f663ffff640 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f663f7fe640 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 -- 172.21.15.19:0/3583617879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 msgr2=0x5652e74d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 -- 172.21.15.19:0/3583617879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 msgr2=0x5652e74fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74fa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 -- 172.21.15.19:0/3583617879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e74e3920 con 0x5652e7138410 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f663f7fe640 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.726+0000 7f6644cea640 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74fabd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6638004af0 tx=0x7f66380099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27642b09a4faf8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f6625ffb640 1 -- 172.21.15.19:0/3583617879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638060070 con 0x5652e7138410 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f6625ffb640 1 -- 172.21.15.19:0/3583617879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638053aa0 con 0x5652e7138410 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f6625ffb640 1 -- 172.21.15.19:0/3583617879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638038450 con 0x5652e7138410 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 -- 172.21.15.19:0/3583617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 msgr2=0x5652e74fabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74fabd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6638004af0 tx=0x7f66380099f0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 -- 172.21.15.19:0/3583617879 shutdown_connections 2024-04-25T10:56:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7138410 0x5652e74fabd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7117730 0x5652e74fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 --2- 172.21.15.19:0/3583617879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 -- 172.21.15.19:0/3583617879 >> 172.21.15.19:0/3583617879 conn(0x5652e712c560 msgr2=0x5652e74d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.727+0000 7f664d601740 1 -- 172.21.15.19:0/3583617879 shutdown_connections 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 -- 172.21.15.19:0/3583617879 wait complete. 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 -- start start 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 Processor -- start 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7117730 0x5652e74e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7138410 0x5652e74e6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66401ac580 con 0x5652e7117730 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66401ad9f0 con 0x5652e7138410 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66401acc50 con 0x5652e710f550 2024-04-25T10:56:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f664d601740 10 client.0 _ll_register_callbacks cb 0x7f66380093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.728+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7117730 0x5652e74e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7117730 0x5652e74e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50292/0 (socket says 172.21.15.19:50292) 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 -- 172.21.15.19:0/1137357540 learned_addr learned my addr 172.21.15.19:0/1137357540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f6644cea640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7138410 0x5652e74e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 -- 172.21.15.19:0/1137357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 msgr2=0x5652e74eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 -- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7138410 msgr2=0x5652e74e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7138410 0x5652e74e6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 -- 172.21.15.19:0/1137357540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e74d83b0 con 0x5652e7117730 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663ffff640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f6644cea640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7138410 0x5652e74e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:56:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f663f7fe640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7117730 0x5652e74e6440 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6634009f60 tx=0x7f6634009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752a8bf4a3ec529a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f661f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f661f7fe640 1 -- 172.21.15.19:0/1137357540 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634026060 con 0x5652e7117730 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f661f7fe640 1 -- 172.21.15.19:0/1137357540 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6634004460 con 0x5652e7117730 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6638053aa0 con 0x5652e7117730 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.729+0000 7f661f7fe640 1 -- 172.21.15.19:0/1137357540 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6634010e80 con 0x5652e7117730 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 10 client.15780 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5652e755cdf0 con 0x5652e7117730 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f661f7fe640 1 -- 172.21.15.19:0/1137357540 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638053aa0 con 0x5652e7117730 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f661f7fe640 1 -- 172.21.15.19:0/1137357540 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f663400f040 con 0x5652e7117730 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 10 client.15780 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 20 client.15780 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70cc630 0x5652e755f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 4 client.15780 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5652e7446580 con 0x5652e70cc630 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.730+0000 7f663ffff640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70cc630 0x5652e755f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.731+0000 7f663ffff640 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70cc630 0x5652e755f5e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5bd11e7a8d46135 server_cookie=9b87fc020b4649a8 in_seq=0 out_seq=0 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.731+0000 7f661f7fe640 10 client.15780 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f661f7fe640 1 -- 172.21.15.19:0/1137357540 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5652e7446580 con 0x5652e70cc630 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f661f7fe640 10 client.15780 handle_command_reply: tid=0 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 client.15780 shutdown 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7117730 msgr2=0x5652e74e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7117730 0x5652e74e6440 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6634009f60 tx=0x7f6634009f90 comp rx=0 tx=0).stop 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 shutdown_connections 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5652e70cc630 0x5652e755f5e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652e7138410 0x5652e74e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652e7117730 0x5652e74e6440 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f661f7fe640 0 client.15780 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 --2- 172.21.15.19:0/1137357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5652e710f550 0x5652e74eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 >> 172.21.15.19:0/1137357540 conn(0x5652e712c560 msgr2=0x5652e74e1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 shutdown_connections 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 1 -- 172.21.15.19:0/1137357540 wait complete. 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.732+0000 7f664d601740 20 client.15780 trim_cache size 0 max 16384 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "11a7cf14-6a54-4e9c-ad07-32203c7a4ef5", 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:56:10.741 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:56:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f66280afc60 msgr2=0x7f66280b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f66280afc60 0x7f66280b2170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66380094e0 tx=0x7f663802a9e0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6640104c50 msgr2=0x7f664019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6640104c50 0x7f664019a710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f663000e9f0 tx=0x7f663000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 shutdown_connections 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f66280afc60 0x7f66280b2170 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f664010e060 0x7f664019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664010b6b0 0x7f664019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 --2- 172.21.15.19:0/4072071434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6640104c50 0x7f664019a710 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 >> 172.21.15.19:0/4072071434 conn(0x7f6640078100 msgr2=0x7f6640101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 shutdown_connections 2024-04-25T10:56:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.733+0000 7f664674d640 1 -- 172.21.15.19:0/4072071434 wait complete. 2024-04-25T10:56:10.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:56:10.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 -- 172.21.15.19:0/614167103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944102af0 msgr2=0x7f3944102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 --2- 172.21.15.19:0/614167103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944102af0 0x7f3944102f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3930009900 tx=0x7f393002a410 comp rx=0 tx=0).stop 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 -- 172.21.15.19:0/614167103 shutdown_connections 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 --2- 172.21.15.19:0/614167103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f394410d380 0x7f394410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 --2- 172.21.15.19:0/614167103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944102af0 0x7f3944102f00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 --2- 172.21.15.19:0/614167103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944108a20 0x7f39441025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 -- 172.21.15.19:0/614167103 >> 172.21.15.19:0/614167103 conn(0x7f3944078100 msgr2=0x7f3944078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 -- 172.21.15.19:0/614167103 shutdown_connections 2024-04-25T10:56:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.909+0000 7f394af3f640 1 -- 172.21.15.19:0/614167103 wait complete. 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 Processor -- start 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 -- start start 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102af0 0x7f394419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944108a20 0x7f394419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f394410d380 0x7f394419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3944112be0 con 0x7f3944108a20 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3944112a60 con 0x7f3944102af0 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f394af3f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3944112d60 con 0x7f394410d380 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f3943fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944108a20 0x7f394419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f3943fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944108a20 0x7f394419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50308/0 (socket says 172.21.15.19:50308) 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f3943fff640 1 -- 172.21.15.19:0/928242549 learned_addr learned my addr 172.21.15.19:0/928242549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.910+0000 7f39494dc640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f394410d380 0x7f394419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3948cdb640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102af0 0x7f394419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3943fff640 1 -- 172.21.15.19:0/928242549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f394410d380 msgr2=0x7f394419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3943fff640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f394410d380 0x7f394419c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3943fff640 1 -- 172.21.15.19:0/928242549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102af0 msgr2=0x7f394419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3943fff640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102af0 0x7f394419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3943fff640 1 -- 172.21.15.19:0/928242549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39441a6910 con 0x7f3944108a20 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3948cdb640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102af0 0x7f394419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3943fff640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944108a20 0x7f394419bb40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f393002a930 tx=0x7f3930004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3941ffb640 1 -- 172.21.15.19:0/928242549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393003f030 con 0x7f3944108a20 2024-04-25T10:56:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3941ffb640 1 -- 172.21.15.19:0/928242549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3930003aa0 con 0x7f3944108a20 2024-04-25T10:56:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39441a6ba0 con 0x7f3944108a20 2024-04-25T10:56:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f3941ffb640 1 -- 172.21.15.19:0/928242549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930035070 con 0x7f3944108a20 2024-04-25T10:56:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.911+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39441a70e0 con 0x7f3944108a20 2024-04-25T10:56:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.914+0000 7f3941ffb640 1 -- 172.21.15.19:0/928242549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3930042030 con 0x7f3944108a20 2024-04-25T10:56:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.914+0000 7f3951df3740 1 Processor -- start 2024-04-25T10:56:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.914+0000 7f3951df3740 1 -- start start 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3951df3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaeafa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3951df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaae64d0 0x5563aaeaffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3951df3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaebc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3951df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5563aae15d80 con 0x5563aaae64d0 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3951df3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563aadcf8b0 con 0x5563aaaee6b0 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3951df3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5563aae11d40 con 0x5563aab0f390 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f39494dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaeafa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3948cdb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaae64d0 0x5563aaeaffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f39494dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaeafa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:51738/0 (socket says 172.21.15.19:51738) 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f39494dc640 1 -- 172.21.15.19:0/2109102909 learned_addr learned my addr 172.21.15.19:0/2109102909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3948cdb640 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaae64d0 0x5563aaeaffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50320/0 (socket says 172.21.15.19:50320) 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3941ffb640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f39340afce0 0x7f39340b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3943fff640 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaebc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f39494dc640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f39340afce0 0x7f39340b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3941ffb640 1 -- 172.21.15.19:0/928242549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f39300ff6f0 con 0x7f3944108a20 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3948cdb640 1 -- 172.21.15.19:0/2109102909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 msgr2=0x5563aaeafa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3948cdb640 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaeafa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3948cdb640 1 -- 172.21.15.19:0/2109102909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 msgr2=0x5563aaebc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3948cdb640 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaebc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f3948cdb640 1 -- 172.21.15.19:0/2109102909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563aada0a40 con 0x5563aaae64d0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.915+0000 7f39494dc640 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaeafa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.916+0000 7f3948cdb640 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaae64d0 0x5563aaeaffa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3944103a20 tx=0x7f393800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c340635f6379af3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.916+0000 7f39494dc640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f39340afce0 0x7f39340b21f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5563aaeacbd0 tx=0x7f393c004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.916+0000 7f392effd640 1 -- 172.21.15.19:0/2109102909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938015070 con 0x5563aaae64d0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.916+0000 7f392effd640 1 -- 172.21.15.19:0/2109102909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393801a910 con 0x5563aaae64d0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.916+0000 7f392effd640 1 -- 172.21.15.19:0/2109102909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938011840 con 0x5563aaae64d0 2024-04-25T10:56:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 -- 172.21.15.19:0/2109102909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaae64d0 msgr2=0x5563aaeaffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaae64d0 0x5563aaeaffa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3944103a20 tx=0x7f393800f730 comp rx=0 tx=0).stop 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 -- 172.21.15.19:0/2109102909 shutdown_connections 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaebc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaae64d0 0x5563aaeaffa0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 --2- 172.21.15.19:0/2109102909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaeafa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 -- 172.21.15.19:0/2109102909 >> 172.21.15.19:0/2109102909 conn(0x5563aab034e0 msgr2=0x5563aaeac210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 -- 172.21.15.19:0/2109102909 shutdown_connections 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 -- 172.21.15.19:0/2109102909 wait complete. 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 -- start start 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.917+0000 7f3951df3740 1 Processor -- start 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3951df3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaeba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3951df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaaee6b0 0x5563aaebab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3951df3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaec04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3951df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5563aaebcfe0 con 0x5563aaaee6b0 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3951df3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563aad9cd10 con 0x5563aaae64d0 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3951df3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5563aaebd160 con 0x5563aab0f390 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3951df3740 10 client.0 _ll_register_callbacks cb 0x5563aadc9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaaee6b0 0x5563aaebab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaaee6b0 0x5563aaebab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50334/0 (socket says 172.21.15.19:50334) 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 -- 172.21.15.19:0/1109237747 learned_addr learned my addr 172.21.15.19:0/1109237747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f39494dc640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaec04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3948cdb640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaeba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 -- 172.21.15.19:0/1109237747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 msgr2=0x5563aaec04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaec04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 -- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 msgr2=0x5563aaeba640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaeba640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 -- 172.21.15.19:0/1109237747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563aabf17d0 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3948cdb640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaeba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3943fff640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaaee6b0 0x5563aaebab80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f39300c7560 tx=0x7f3930038fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2d5278804fdcb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.918+0000 7f3923fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3923fff640 1 -- 172.21.15.19:0/1109237747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930049070 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f393801a910 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3923fff640 1 -- 172.21.15.19:0/1109237747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f39300c3e50 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3923fff640 1 -- 172.21.15.19:0/1109237747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f39300c7370 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 10 client.15792 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5563aab273f0 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3923fff640 1 -- 172.21.15.19:0/1109237747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393801a910 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3923fff640 1 -- 172.21.15.19:0/1109237747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f39300c3af0 con 0x5563aaaee6b0 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 10 client.15792 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:10.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 20 client.15792 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:10.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaacdf60 0x5563aaecc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 4 client.15792 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:10.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563aae23990 con 0x5563aaacdf60 2024-04-25T10:56:10.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.919+0000 7f3948cdb640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaacdf60 0x5563aaecc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.920+0000 7f3923fff640 10 client.15792 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.920+0000 7f3948cdb640 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaacdf60 0x5563aaecc750 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645fe80497750e01 server_cookie=610bfe1c308173e2 in_seq=0 out_seq=0 2024-04-25T10:56:10.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3923fff640 1 -- 172.21.15.19:0/1109237747 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563aae23990 con 0x5563aaacdf60 2024-04-25T10:56:10.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3923fff640 10 client.15792 handle_command_reply: tid=0 2024-04-25T10:56:10.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 client.15792 shutdown 2024-04-25T10:56:10.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaaee6b0 msgr2=0x5563aaebab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaaee6b0 0x5563aaebab80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f39300c7560 tx=0x7f3930038fd0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 shutdown_connections 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaacdf60 0x5563aaecc750 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aab0f390 0x5563aaec04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aaaee6b0 0x5563aaebab80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3923fff640 0 client.15792 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 --2- 172.21.15.19:0/1109237747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaeba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 >> 172.21.15.19:0/1109237747 conn(0x5563aab034e0 msgr2=0x5563aaeb1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 shutdown_connections 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 1 -- 172.21.15.19:0/1109237747 wait complete. 2024-04-25T10:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.921+0000 7f3951df3740 20 client.15792 trim_cache size 0 max 16384 2024-04-25T10:56:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 Processor -- start 2024-04-25T10:56:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 -- start start 2024-04-25T10:56:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaae64d0 0x5563aaeb9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaeb5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaeb1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39441ab7b0 con 0x5563aab0f390 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39441ae030 con 0x5563aaaee6b0 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.942+0000 7f3951df3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f39441abe20 con 0x5563aaae64d0 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaae64d0 0x5563aaeb9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f39494dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaeb1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaae64d0 0x5563aaeb9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:51762/0 (socket says 172.21.15.19:51762) 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 -- 172.21.15.19:0/660530666 learned_addr learned my addr 172.21.15.19:0/660530666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3943fff640 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaeb5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 -- 172.21.15.19:0/660530666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 msgr2=0x5563aaeb5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaeb5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 -- 172.21.15.19:0/660530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 msgr2=0x5563aaeb1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaeb1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 -- 172.21.15.19:0/660530666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563aaae70f0 con 0x5563aaae64d0 2024-04-25T10:56:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f39494dc640 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaeb1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f3948cdb640 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaae64d0 0x5563aaeb9370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39380114b0 tx=0x7f393800b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a79647b258abf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f392e7fc640 1 -- 172.21.15.19:0/660530666 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938015070 con 0x5563aaae64d0 2024-04-25T10:56:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f392e7fc640 1 -- 172.21.15.19:0/660530666 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3938026e50 con 0x5563aaae64d0 2024-04-25T10:56:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.943+0000 7f392e7fc640 1 -- 172.21.15.19:0/660530666 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938026480 con 0x5563aaae64d0 2024-04-25T10:56:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 -- 172.21.15.19:0/660530666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaae64d0 msgr2=0x5563aaeb9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaae64d0 0x5563aaeb9370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39380114b0 tx=0x7f393800b5a0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 -- 172.21.15.19:0/660530666 shutdown_connections 2024-04-25T10:56:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaeb1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaaee6b0 0x5563aaeb5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 --2- 172.21.15.19:0/660530666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaae64d0 0x5563aaeb9370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 -- 172.21.15.19:0/660530666 >> 172.21.15.19:0/660530666 conn(0x5563aab034e0 msgr2=0x5563aaeb1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 -- 172.21.15.19:0/660530666 shutdown_connections 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.944+0000 7f3951df3740 1 -- 172.21.15.19:0/660530666 wait complete. 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 -- start start 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 Processor -- start 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaec6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaaee6b0 0x5563aaec1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaec2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39441ac4f0 con 0x5563aab0f390 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39441ad960 con 0x5563aaae64d0 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f39441acbc0 con 0x5563aaaee6b0 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f3951df3740 10 client.0 _ll_register_callbacks cb 0x7f3930005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.945+0000 7f39494dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaec2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaec2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50344/0 (socket says 172.21.15.19:50344) 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f3948cdb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaec6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 -- 172.21.15.19:0/1327301705 learned_addr learned my addr 172.21.15.19:0/1327301705 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f3943fff640 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaaee6b0 0x5563aaec1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 -- 172.21.15.19:0/1327301705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaaee6b0 msgr2=0x5563aaec1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaaee6b0 0x5563aaec1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 -- 172.21.15.19:0/1327301705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 msgr2=0x5563aaec6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaec6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 -- 172.21.15.19:0/1327301705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563aadd64b0 con 0x5563aab0f390 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f39494dc640 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaec2520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f393c004530 tx=0x7f393c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96caed25ee056534 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f3923fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f3923fff640 1 -- 172.21.15.19:0/1327301705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c01b050 con 0x5563aab0f390 2024-04-25T10:56:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f3923fff640 1 -- 172.21.15.19:0/1327301705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f393c025c90 con 0x5563aab0f390 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.946+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3938026e50 con 0x5563aab0f390 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3923fff640 1 -- 172.21.15.19:0/1327301705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f393c02f420 con 0x5563aab0f390 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 10 client.15798 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5563aaf345b0 con 0x5563aab0f390 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3923fff640 1 -- 172.21.15.19:0/1327301705 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3938026e50 con 0x5563aab0f390 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3923fff640 1 -- 172.21.15.19:0/1327301705 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f393c024390 con 0x5563aab0f390 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 10 client.15798 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 20 client.15798 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaaa33a0 0x5563aaf36d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 4 client.15798 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563aae1d800 con 0x5563aaaa33a0 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.947+0000 7f3948cdb640 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaaa33a0 0x5563aaf36d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3923fff640 10 client.15798 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3948cdb640 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaaa33a0 0x5563aaf36d80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea952fc02692d042 server_cookie=fb3c9b62a30c6776 in_seq=0 out_seq=0 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3923fff640 1 -- 172.21.15.19:0/1327301705 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5563aae1d800 con 0x5563aaaa33a0 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3923fff640 10 client.15798 handle_command_reply: tid=0 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 client.15798 shutdown 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 msgr2=0x5563aaec2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaec2520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f393c004530 tx=0x7f393c004a40 comp rx=0 tx=0).stop 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 shutdown_connections 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5563aaaa33a0 0x5563aaf36d80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5563aab0f390 0x5563aaec2520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5563aaaee6b0 0x5563aaec1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 --2- 172.21.15.19:0/1327301705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563aaae64d0 0x5563aaec6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 >> 172.21.15.19:0/1327301705 conn(0x5563aab034e0 msgr2=0x5563aaebb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3923fff640 0 client.15798 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 shutdown_connections 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 1 -- 172.21.15.19:0/1327301705 wait complete. 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.948+0000 7f3951df3740 20 client.15798 trim_cache size 0 max 16384 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stdout: "11a7cf14-6a54-4e9c-ad07-32203c7a4ef5": { 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "11a7cf14-6a54-4e9c-ad07-32203c7a4ef5", 2024-04-25T10:56:10.955 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:56:10.956 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:56:10.956 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:56:10.956 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:56:10.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f39340afce0 msgr2=0x7f39340b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f39340afce0 0x7f39340b21f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5563aaeacbd0 tx=0x7f393c004ab0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944108a20 msgr2=0x7f394419bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944108a20 0x7f394419bb40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f393002a930 tx=0x7f3930004270 comp rx=0 tx=0).stop 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 shutdown_connections 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f39340afce0 0x7f39340b21f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f394410d380 0x7f394419c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3944108a20 0x7f394419bb40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 --2- 172.21.15.19:0/928242549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102af0 0x7f394419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 >> 172.21.15.19:0/928242549 conn(0x7f3944078100 msgr2=0x7f39441040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 shutdown_connections 2024-04-25T10:56:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:10.949+0000 7f394af3f640 1 -- 172.21.15.19:0/928242549 wait complete. 2024-04-25T10:56:10.961 INFO:tasks.cephfs.filesystem:scrub status for tag:11a7cf14-6a54-4e9c-ad07-32203c7a4ef5 - {'path': '/', 'tag': '11a7cf14-6a54-4e9c-ad07-32203c7a4ef5', 'options': 'recursive,force'} 2024-04-25T10:56:11.000 INFO:tasks.workunit.client.0.smithi019.stdout:.......... 2024-04-25T10:56:11.000 INFO:tasks.workunit.client.0.smithi019.stdout:do_o_direct_reads: SUCCESS 2024-04-25T10:56:11.005 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T10:56:11.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T10:56:11.075 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-04-25T10:56:11.076 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-04-25T10:56:11.135 INFO:tasks.workunit.client.0.smithi019.stdout:test read from hole 2024-04-25T10:56:11.136 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test read from hole' 2024-04-25T10:56:11.136 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-04-25T10:56:11.137 INFO:tasks.workunit.client.0.smithi019.stderr:0+0 records in 2024-04-25T10:56:11.137 INFO:tasks.workunit.client.0.smithi019.stderr:0+0 records out 2024-04-25T10:56:11.137 INFO:tasks.workunit.client.0.smithi019.stderr:0 bytes copied, 3.5658e-05 s, 0.0 kB/s 2024-04-25T10:56:11.137 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-04-25T10:56:11.139 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records in 2024-04-25T10:56:11.139 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records out 2024-04-25T10:56:11.139 INFO:tasks.workunit.client.0.smithi019.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00136038 s, 753 kB/s 2024-04-25T10:56:11.140 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-04-25T10:56:11.140 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records in 2024-04-25T10:56:11.140 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records out 2024-04-25T10:56:11.140 INFO:tasks.workunit.client.0.smithi019.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 5.0087e-05 s, 20.4 MB/s 2024-04-25T10:56:11.140 INFO:tasks.workunit.client.0.smithi019.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-04-25T10:56:11.141 INFO:tasks.workunit.client.0.smithi019.stdout:other thing 2024-04-25T10:56:11.141 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'other thing' 2024-04-25T10:56:11.141 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-04-25T10:56:11.142 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records in 2024-04-25T10:56:11.142 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records out 2024-04-25T10:56:11.142 INFO:tasks.workunit.client.0.smithi019.stderr:500 bytes copied, 4.2123e-05 s, 11.9 MB/s 2024-04-25T10:56:11.142 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-04-25T10:56:11.149 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records in 2024-04-25T10:56:11.149 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records out 2024-04-25T10:56:11.149 INFO:tasks.workunit.client.0.smithi019.stderr:500 bytes copied, 0.00567373 s, 88.1 kB/s 2024-04-25T10:56:11.149 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-04-25T10:56:11.155 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records in 2024-04-25T10:56:11.155 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records out 2024-04-25T10:56:11.155 INFO:tasks.workunit.client.0.smithi019.stderr:500 bytes copied, 0.00536138 s, 93.3 kB/s 2024-04-25T10:56:11.155 INFO:tasks.workunit.client.0.smithi019.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-04-25T10:56:11.156 INFO:tasks.workunit.client.0.smithi019.stdout:OK 2024-04-25T10:56:11.157 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo OK 2024-04-25T10:56:11.157 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T10:56:11.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T10:56:11.218 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:11 smithi019 ceph-mon[29209]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:11.218 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:11 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:56:11.218 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:11 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:11.218 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:11 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:56:11.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:11 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:11.233 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-04-25T10:56:11.234 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-04-25T10:56:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:writing first 3 bytes of 10k file 2024-04-25T10:56:11.303 INFO:tasks.workunit.client.0.smithi019.stdout:reading O_DIRECT 2024-04-25T10:56:11.303 INFO:tasks.workunit.client.0.smithi019.stdout:got 10000 2024-04-25T10:56:11.303 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T10:56:11.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T10:56:11.376 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-04-25T10:56:11.376 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-04-25T10:56:11.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:11 smithi169 ceph-mon[33049]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:11.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:11 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:56:11.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:11 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:11.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:11 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:56:11.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:11 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:11.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:11 smithi112 ceph-mon[32799]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-04-25T10:56:11.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:11 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:56:11.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:11 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:11.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:11 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:56:11.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:11 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:12.481 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:12 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:56:12.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:12 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:56:12.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:12 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:writing pattern 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-25T10:56:12.838 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:12.839 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:12.840 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:12.841 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:12.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:12.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:12.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:12.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:12.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:12.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:12.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:12.858 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:12.859 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:12.860 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:12.861 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:12.862 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:12.863 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:12.864 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:12.864 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:12.864 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:12.864 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:12.864 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:12.864 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:12.864 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:12.868 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:12.868 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:12.868 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:12.868 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:12.868 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:12.868 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:12.868 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:12.869 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:12.870 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:12.871 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:12.872 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:12.873 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:12.874 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:12.874 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:12.874 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:12.874 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:12.874 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:12.874 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:12.874 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:12.880 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:12.881 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:12.882 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:12.883 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-25T10:56:12.884 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:12.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:12.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:12.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:12.898 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:12.901 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:12.902 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:12.903 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:12.904 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:12.904 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:12.904 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:12.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:12.921 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:12.921 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:12.924 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:12.924 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:12.926 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:12.930 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-25T10:56:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:12.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:12.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:12.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:12.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:12.956 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:12.956 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:12.956 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:12.956 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:12.957 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:12.958 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:12.961 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:12.973 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:12.974 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:12.976 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:12.977 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:12.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:12.987 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:12.990 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:12.991 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:12.992 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:12.993 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:13.003 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:13.004 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-25T10:56:13.005 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:13.006 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:13.007 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:13.008 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:13.009 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:13.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:13.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:13.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:13.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:13.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:13.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:13.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:13.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:13.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:13.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:13.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:13.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:13.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:13.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:13.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:13.038 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:13.039 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:13.040 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:13.041 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:13.042 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:13.043 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:13.044 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:13.052 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:13.053 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:13.054 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:13.055 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:13.056 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:13.057 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:13.058 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:13.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:13.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:13.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:13.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:13.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:13.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-04-25T10:56:13.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:13.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:13.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:13.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:13.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:13.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:13.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:13.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:13.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:13.088 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:13.089 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:13.090 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:13.091 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:13.092 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:13.093 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:13.094 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:13.101 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:13.102 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:13.103 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:13.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:13.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:13.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:13.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:13.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:13.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:13.113 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:13.114 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:13.115 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:13.116 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:13.125 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-25T10:56:13.126 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-04-25T10:56:13.127 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:13.128 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:13.129 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:13.130 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:13.145 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:13.145 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:13.145 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:13.145 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:13.145 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:13.145 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:13.146 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:13.147 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:13.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:13.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:13.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:13.157 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:13.159 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:13.160 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:13.161 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:13.162 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:13.166 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:13.167 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:13.168 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:13.169 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:13.170 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:13.171 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:13.172 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:13.172 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-04-25T10:56:13.182 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:13.183 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:13.184 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:13.185 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:13.186 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:13.187 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:13.200 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:13.200 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:13.200 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:13.201 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:13.202 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:13.203 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:13.204 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:13.205 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:13.206 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:13.207 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:13.210 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:13.210 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:13.210 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:13.210 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:13.210 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:13.211 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:13.212 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:13.213 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:13.214 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:13.215 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:13.216 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:13.220 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:13.220 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:13.220 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:13.220 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:13.220 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:13.220 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:13.220 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:13.221 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:13.222 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:13.223 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:13.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-04-25T10:56:13.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-04-25T10:56:13.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-25T10:56:13.238 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-04-25T10:56:13.238 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-25T10:56:13.238 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-04-25T10:56:13.238 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-25T10:56:13.238 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-04-25T10:56:13.238 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-25T10:56:13.238 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:13.240 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:13.242 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:13.244 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:13.244 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:13.244 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:13.257 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:13.259 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:13.260 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:13.261 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:13.263 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:13.268 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:13.268 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:13.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:13.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:13.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:13.436 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:13.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:13.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:13.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-25T10:56:13.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:13.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:13 smithi169 ceph-mon[33049]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.00k op/s 2024-04-25T10:56:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:13 smithi112 ceph-mon[32799]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.00k op/s 2024-04-25T10:56:13.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:13 smithi019 ceph-mon[29209]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.00k op/s 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:14.020 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:14.021 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:14.022 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-25T10:56:14.023 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:14.024 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:14.025 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:14.026 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-25T10:56:14.026 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:14.026 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:14.026 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:14.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-25T10:56:14.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:14.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:14.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:14.497 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:14.498 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-25T10:56:14.498 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:14.498 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:14.498 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:14.498 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-25T10:56:14.498 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:14.498 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:14.499 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:14.500 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:14.501 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:14.502 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:14.503 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:14.504 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:14.505 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:14.506 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-25T10:56:14.902 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-25T10:56:14.903 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:14.904 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:14.905 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:14.906 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:14.907 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:14.908 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:14.909 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:14.909 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-25T10:56:14.909 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:14.909 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:15.431 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:15.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:15.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:15.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-25T10:56:15.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:15.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:15.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:15 smithi169 ceph-mon[33049]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.00k op/s 2024-04-25T10:56:15.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:15 smithi112 ceph-mon[32799]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.00k op/s 2024-04-25T10:56:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:15 smithi019 ceph-mon[29209]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.00k op/s 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:16.062 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-25T10:56:16.063 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:16.064 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:16.065 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:16.066 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:16.067 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:16.068 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-25T10:56:16.447 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:16.447 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:16.448 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:16.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-25T10:56:16.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:16.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:16.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:16.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:16.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:16.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-25T10:56:16.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:16.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:16.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:16.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-25T10:56:16.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:16.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:16.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:16.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-25T10:56:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:17.402 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:17.402 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:17.402 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-25T10:56:17.402 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:17.403 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-25T10:56:17.404 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:17.405 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:17.406 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:17.407 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-25T10:56:17.408 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:17.408 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:17.408 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:17.408 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-25T10:56:17.408 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:17 smithi169 ceph-mon[33049]: pgmap v56: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 2.4 MiB/s wr, 3.27k op/s 2024-04-25T10:56:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:17 smithi112 ceph-mon[32799]: pgmap v56: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 2.4 MiB/s wr, 3.27k op/s 2024-04-25T10:56:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:17 smithi019 ceph-mon[29209]: pgmap v56: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 2.4 MiB/s wr, 3.27k op/s 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-25T10:56:17.935 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-25T10:56:17.936 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:17.937 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:17.938 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:17.939 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:17.940 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:17.941 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:18.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-25T10:56:18.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:18.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:18.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:18.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:18.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:18.738 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-25T10:56:18.738 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:18.739 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-25T10:56:18.740 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:18.741 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:18.742 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:18.743 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:18.744 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-25T10:56:19.274 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:19.274 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:19.274 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:19.274 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-25T10:56:19.274 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:19.275 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:19.276 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:19.277 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-25T10:56:19.278 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:19.279 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:19.280 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-25T10:56:19.280 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:19.280 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:19.280 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:19.280 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-25T10:56:19.280 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:19.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:19 smithi169 ceph-mon[33049]: pgmap v57: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.0 MiB/s wr, 1.88k op/s 2024-04-25T10:56:19.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:19 smithi112 ceph-mon[32799]: pgmap v57: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.0 MiB/s wr, 1.88k op/s 2024-04-25T10:56:19.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:19 smithi019 ceph-mon[29209]: pgmap v57: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.0 MiB/s wr, 1.88k op/s 2024-04-25T10:56:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-25T10:56:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-25T10:56:19.726 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:19.727 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:19.728 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:19.729 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-25T10:56:19.729 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:20.173 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:20.173 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:20.173 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-25T10:56:20.173 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:20.173 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:20.173 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:20.174 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-25T10:56:20.175 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:20.176 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:20.177 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:20.178 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:20.179 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-25T10:56:20.179 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:20.179 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:20.179 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:20.179 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-25T10:56:20.179 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:20.179 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:20.625 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:20.626 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:20.627 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-25T10:56:20.628 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:20.629 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-25T10:56:20.630 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:20.631 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:20.631 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:21.201 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-25T10:56:21.202 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:21.203 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:21.204 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:21.205 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:21.206 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:21.207 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:21.207 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-25T10:56:21.412 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:21 smithi019 ceph-mon[29209]: pgmap v58: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.0 MiB/s wr, 1.88k op/s 2024-04-25T10:56:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:21 smithi169 ceph-mon[33049]: pgmap v58: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.0 MiB/s wr, 1.88k op/s 2024-04-25T10:56:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:21 smithi112 ceph-mon[32799]: pgmap v58: 97 pgs: 97 active+clean; 176 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.0 MiB/s wr, 1.88k op/s 2024-04-25T10:56:21.563 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-25T10:56:21.564 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:21.565 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:21.566 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:21.567 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-25T10:56:21.568 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:21.569 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:21.569 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:21.569 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-25T10:56:21.569 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:21.569 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:21.569 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:21.569 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-25T10:56:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 -- 172.21.15.19:0/1286522150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 msgr2=0x7fada4102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 --2- 172.21.15.19:0/1286522150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 0x7fada4102610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad94009900 tx=0x7fad9402a410 comp rx=0 tx=0).stop 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 -- 172.21.15.19:0/1286522150 shutdown_connections 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 --2- 172.21.15.19:0/1286522150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada41031c0 0x7fada410f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 --2- 172.21.15.19:0/1286522150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4108f00 0x7fada4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 --2- 172.21.15.19:0/1286522150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 0x7fada4102610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 -- 172.21.15.19:0/1286522150 >> 172.21.15.19:0/1286522150 conn(0x7fada40781b0 msgr2=0x7fada40785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 -- 172.21.15.19:0/1286522150 shutdown_connections 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.743+0000 7fadaa269640 1 -- 172.21.15.19:0/1286522150 wait complete. 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.744+0000 7fadaa269640 1 Processor -- start 2024-04-25T10:56:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.744+0000 7fadaa269640 1 -- start start 2024-04-25T10:56:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fadaa269640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada41031c0 0x7fada419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fadaa269640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 0x7fada419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fadaa269640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada4108f00 0x7fada419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fadaa269640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fada4112c60 con 0x7fada41031c0 2024-04-25T10:56:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fadaa269640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fada4112ae0 con 0x7fada4108a60 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fadaa269640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fada4112de0 con 0x7fada4108f00 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada41031c0 0x7fada419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada41031c0 0x7fada419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55730/0 (socket says 172.21.15.19:55730) 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 -- 172.21.15.19:0/323381463 learned_addr learned my addr 172.21.15.19:0/323381463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada37fe640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 0x7fada419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada8806640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada4108f00 0x7fada419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 -- 172.21.15.19:0/323381463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada4108f00 msgr2=0x7fada419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada4108f00 0x7fada419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 -- 172.21.15.19:0/323381463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 msgr2=0x7fada419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 0x7fada419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 -- 172.21.15.19:0/323381463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada419f5b0 con 0x7fada41031c0 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada37fe640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 0x7fada419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada3fff640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada41031c0 0x7fada419a750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fad9402a930 tx=0x7fad94031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada17fa640 1 -- 172.21.15.19:0/323381463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9403f030 con 0x7fada41031c0 2024-04-25T10:56:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada419f840 con 0x7fada41031c0 2024-04-25T10:56:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada17fa640 1 -- 172.21.15.19:0/323381463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad94004490 con 0x7fada41031c0 2024-04-25T10:56:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.745+0000 7fada17fa640 1 -- 172.21.15.19:0/323381463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94035070 con 0x7fada41031c0 2024-04-25T10:56:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.746+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada419fd30 con 0x7fada41031c0 2024-04-25T10:56:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.748+0000 7fada17fa640 1 -- 172.21.15.19:0/323381463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fad94042030 con 0x7fada41031c0 2024-04-25T10:56:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 Processor -- start 2024-04-25T10:56:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 -- start start 2024-04-25T10:56:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbbb4d0 0x557ebff7c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ebfee2460 con 0x557ebfbb32f0 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557ebfe9c590 con 0x557ebfbdc1b0 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fadb11f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557ebfede420 con 0x557ebfbbb4d0 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fada3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fada8806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fada37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbbb4d0 0x557ebff7c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fada8806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff7baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:32814/0 (socket says 172.21.15.19:32814) 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fada3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff7c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55736/0 (socket says 172.21.15.19:55736) 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fada8806640 1 -- 172.21.15.19:0/3371703214 learned_addr learned my addr 172.21.15.19:0/3371703214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.749+0000 7fada17fa640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fad8c0afc50 0x7fad8c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 -- 172.21.15.19:0/3371703214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbbb4d0 msgr2=0x557ebff7c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 --2- 172.21.15.19:0/3371703214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbbb4d0 0x557ebff7c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 -- 172.21.15.19:0/3371703214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 msgr2=0x557ebff7c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 --2- 172.21.15.19:0/3371703214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff7c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 -- 172.21.15.19:0/3371703214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ebfea90d0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada17fa640 1 -- 172.21.15.19:0/323381463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fad940ff310 con 0x7fada41031c0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fad8c0afc50 0x7fad8c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 --2- 172.21.15.19:0/3371703214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff7baf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011590 tx=0x7fad9c011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d0dd78ddb7607a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fad8a7fc640 1 -- 172.21.15.19:0/3371703214 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c017050 con 0x557ebfbdc1b0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fad8a7fc640 1 -- 172.21.15.19:0/3371703214 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c00fcf0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fada8806640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fad8c0afc50 0x7fad8c0b2160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fad9c022000 tx=0x7fad9c023000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.750+0000 7fad8a7fc640 1 -- 172.21.15.19:0/3371703214 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c01b9a0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 -- 172.21.15.19:0/3371703214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 msgr2=0x557ebff7baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3371703214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff7baf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011590 tx=0x7fad9c011a50 comp rx=0 tx=0).stop 2024-04-25T10:56:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 -- 172.21.15.19:0/3371703214 shutdown_connections 2024-04-25T10:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3371703214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbbb4d0 0x557ebff7c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3371703214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff7c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3371703214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff7baf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 -- 172.21.15.19:0/3371703214 >> 172.21.15.19:0/3371703214 conn(0x557ebfbd0300 msgr2=0x557ebff782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 -- 172.21.15.19:0/3371703214 shutdown_connections 2024-04-25T10:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.751+0000 7fadb11f1740 1 -- 172.21.15.19:0/3371703214 wait complete. 2024-04-25T10:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 -- start start 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 Processor -- start 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbbb4d0 0x557ebff86720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ebff88c20 con 0x557ebfbbb4d0 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557ebfe69b60 con 0x557ebfbdc1b0 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.752+0000 7fadb11f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557ebff88da0 con 0x557ebfbb32f0 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fadb11f1740 10 client.0 _ll_register_callbacks cb 0x557ebfe8e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbbb4d0 0x557ebff86720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff8c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:32824/0 (socket says 172.21.15.19:32824) 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbbb4d0 0x557ebff86720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55738/0 (socket says 172.21.15.19:55738) 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 -- 172.21.15.19:0/2777201451 learned_addr learned my addr 172.21.15.19:0/2777201451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada3fff640 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 -- 172.21.15.19:0/2777201451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 msgr2=0x557ebff861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 -- 172.21.15.19:0/2777201451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbbb4d0 msgr2=0x557ebff86720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbbb4d0 0x557ebff86720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 -- 172.21.15.19:0/2777201451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ebfbf4210 con 0x557ebfbdc1b0 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada37fe640 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbbb4d0 0x557ebff86720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fada8806640 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff8c180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011560 tx=0x7fad9c01b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478fb90f3b3f7b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fad837fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fad837fe640 1 -- 172.21.15.19:0/2777201451 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c035060 con 0x557ebfbdc1b0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c00fcf0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.753+0000 7fad837fe640 1 -- 172.21.15.19:0/2777201451 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fad9c004bf0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fad837fe640 1 -- 172.21.15.19:0/2777201451 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fad9c017020 con 0x557ebfbdc1b0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 10 client.25469 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557ebfcc0620 con 0x557ebfbdc1b0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fad837fe640 1 -- 172.21.15.19:0/2777201451 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c00fcf0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fad837fe640 1 -- 172.21.15.19:0/2777201451 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fad9c01a070 con 0x557ebfbdc1b0 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 10 client.25469 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 20 client.25469 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb9ad80 0x557ebff986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 4 client.25469 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ebfef0070 con 0x557ebfb9ad80 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.754+0000 7fada3fff640 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb9ad80 0x557ebff986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.755+0000 7fad837fe640 10 client.25469 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.755+0000 7fada3fff640 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb9ad80 0x557ebff986c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a516bfc85cd3233 server_cookie=74536a0a4f61c61a in_seq=0 out_seq=0 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fad837fe640 1 -- 172.21.15.19:0/2777201451 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557ebfef0070 con 0x557ebfb9ad80 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fad837fe640 10 client.25469 handle_command_reply: tid=0 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 client.25469 shutdown 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 msgr2=0x557ebff8c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff8c180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011560 tx=0x7fad9c01b9b0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 shutdown_connections 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb9ad80 0x557ebff986c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbdc1b0 0x557ebff8c180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbbb4d0 0x557ebff86720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fad837fe640 0 client.25469 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 --2- 172.21.15.19:0/2777201451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 >> 172.21.15.19:0/2777201451 conn(0x557ebfbd0300 msgr2=0x557ebff7b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 shutdown_connections 2024-04-25T10:56:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 1 -- 172.21.15.19:0/2777201451 wait complete. 2024-04-25T10:56:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.756+0000 7fadb11f1740 20 client.25469 trim_cache size 0 max 16384 2024-04-25T10:56:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 Processor -- start 2024-04-25T10:56:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 -- start start 2024-04-25T10:56:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff8c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff9dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbdc1b0 0x557ebff9e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fada41ad960 con 0x557ebfbb32f0 2024-04-25T10:56:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fada41b01e0 con 0x557ebfbbb4d0 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fadb11f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fada41adfd0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff8c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff8c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55744/0 (socket says 172.21.15.19:55744) 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 -- 172.21.15.19:0/3542782916 learned_addr learned my addr 172.21.15.19:0/3542782916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada37fe640 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff9dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada8806640 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbdc1b0 0x557ebff9e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 -- 172.21.15.19:0/3542782916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbdc1b0 msgr2=0x557ebff9e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbdc1b0 0x557ebff9e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 -- 172.21.15.19:0/3542782916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 msgr2=0x557ebff9dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff9dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.772+0000 7fada3fff640 1 -- 172.21.15.19:0/3542782916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ebff8c550 con 0x557ebfbb32f0 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.773+0000 7fada3fff640 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff8c010 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fad94005ab0 tx=0x7fad940c3d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73130e4d03d5478a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.773+0000 7fad89ffb640 1 -- 172.21.15.19:0/3542782916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94049040 con 0x557ebfbb32f0 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.773+0000 7fad89ffb640 1 -- 172.21.15.19:0/3542782916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad940c70a0 con 0x557ebfbb32f0 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.773+0000 7fad89ffb640 1 -- 172.21.15.19:0/3542782916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad940d0340 con 0x557ebfbb32f0 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 -- 172.21.15.19:0/3542782916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 msgr2=0x557ebff8c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff8c010 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fad94005ab0 tx=0x7fad940c3d10 comp rx=0 tx=0).stop 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 -- 172.21.15.19:0/3542782916 shutdown_connections 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbdc1b0 0x557ebff9e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff9dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 --2- 172.21.15.19:0/3542782916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbb32f0 0x557ebff8c010 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 -- 172.21.15.19:0/3542782916 >> 172.21.15.19:0/3542782916 conn(0x557ebfbd0300 msgr2=0x557ebff848a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 -- 172.21.15.19:0/3542782916 shutdown_connections 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 -- 172.21.15.19:0/3542782916 wait complete. 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 -- start start 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 Processor -- start 2024-04-25T10:56:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff92ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff8de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbdc1b0 0x557ebff8e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.774+0000 7fadb11f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fada41ae6a0 con 0x557ebfbdc1b0 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fadb11f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fada41afb10 con 0x557ebfbbb4d0 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fadb11f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fada41aed70 con 0x557ebfbb32f0 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fadb11f1740 10 client.0 _ll_register_callbacks cb 0x7fad940c3c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada8806640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbdc1b0 0x557ebff8e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff92ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff92ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:53126/0 (socket says 172.21.15.19:53126) 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff8de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada8806640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbdc1b0 0x557ebff8e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55760/0 (socket says 172.21.15.19:55760) 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 -- 172.21.15.19:0/1431076150 learned_addr learned my addr 172.21.15.19:0/1431076150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 -- 172.21.15.19:0/1431076150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 msgr2=0x557ebff8de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff8de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 -- 172.21.15.19:0/1431076150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbdc1b0 msgr2=0x557ebff8e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbdc1b0 0x557ebff8e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 -- 172.21.15.19:0/1431076150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ebff8e9e0 con 0x557ebfbb32f0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada8806640 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbdc1b0 0x557ebff8e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fada3fff640 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff92ac0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad94031440 tx=0x7fad9402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22235cc1f6e4897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fad837fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fad837fe640 1 -- 172.21.15.19:0/1431076150 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94049030 con 0x557ebfbb32f0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fad837fe640 1 -- 172.21.15.19:0/1431076150 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fad940fcd10 con 0x557ebfbb32f0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.775+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad940c70a0 con 0x557ebfbb32f0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fad837fe640 1 -- 172.21.15.19:0/1431076150 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fad940d0340 con 0x557ebfbb32f0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 10 client.25420 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557ec00006a0 con 0x557ebfbb32f0 2024-04-25T10:56:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fad837fe640 1 -- 172.21.15.19:0/1431076150 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad940c70a0 con 0x557ebfbb32f0 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fad837fe640 1 -- 172.21.15.19:0/1431076150 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fad9c01bce0 con 0x557ebfbb32f0 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 10 client.25420 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 10 client.25420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 20 client.25420 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb701c0 0x557ec0002d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 4 client.25420 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ebfee9ee0 con 0x557ebfb701c0 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.776+0000 7fada37fe640 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb701c0 0x557ec0002d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.777+0000 7fada37fe640 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb701c0 0x557ec0002d70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dd69a289126f6b server_cookie=ed261cdf0dce3ada in_seq=0 out_seq=0 2024-04-25T10:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.777+0000 7fad837fe640 10 client.25420 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fad837fe640 1 -- 172.21.15.19:0/1431076150 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ebfee9ee0 con 0x557ebfb701c0 2024-04-25T10:56:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fad837fe640 10 client.25420 handle_command_reply: tid=0 2024-04-25T10:56:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 client.25420 shutdown 2024-04-25T10:56:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 msgr2=0x557ebff92ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff92ac0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad94031440 tx=0x7fad9402a410 comp rx=0 tx=0).stop 2024-04-25T10:56:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 shutdown_connections 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557ebfb701c0 0x557ec0002d70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ebfbdc1b0 0x557ebff8e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557ebfbbb4d0 0x557ebff8de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fad837fe640 0 client.25420 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 --2- 172.21.15.19:0/1431076150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ebfbb32f0 0x557ebff92ac0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 >> 172.21.15.19:0/1431076150 conn(0x557ebfbd0300 msgr2=0x557ebff7f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 shutdown_connections 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 1 -- 172.21.15.19:0/1431076150 wait complete. 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.842+0000 7fadb11f1740 20 client.25420 trim_cache size 0 max 16384 2024-04-25T10:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T10:56:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T10:56:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T10:56:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fad8c0afc50 msgr2=0x7fad8c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fad8c0afc50 0x7fad8c0b2160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fad9c022000 tx=0x7fad9c023000 comp rx=0 tx=0).stop 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada41031c0 msgr2=0x7fada419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada41031c0 0x7fada419a750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fad9402a930 tx=0x7fad94031a20 comp rx=0 tx=0).stop 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 shutdown_connections 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fad8c0afc50 0x7fad8c0b2160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada4108f00 0x7fada419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada4108a60 0x7fada419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 --2- 172.21.15.19:0/323381463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada41031c0 0x7fada419a750 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 >> 172.21.15.19:0/323381463 conn(0x7fada40781b0 msgr2=0x7fada406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 shutdown_connections 2024-04-25T10:56:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T10:56:21.843+0000 7fadaa269640 1 -- 172.21.15.19:0/323381463 wait complete. 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:21.994 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:21.995 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-25T10:56:21.996 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:21.997 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:21.998 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-25T10:56:21.999 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:22.535 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:22.535 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-25T10:56:22.536 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:22.537 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:22.538 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-25T10:56:22.539 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-25T10:56:22.540 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:22.541 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:22.541 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:22.541 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-25T10:56:22.541 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:22.541 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:23.107 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:23.107 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-25T10:56:23.107 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:23.107 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:23.108 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:23.109 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:23.110 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:23.111 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:23.112 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:23.113 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:23.113 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-25T10:56:23.113 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:23.113 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:23.113 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:23.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:23.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:23.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:23.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:23.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:23.495 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-25T10:56:23.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:23.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:23.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:23.496 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-25T10:56:23.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:23 smithi169 ceph-mon[33049]: pgmap v59: 97 pgs: 97 active+clean; 381 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.7 MiB/s wr, 2.09k op/s 2024-04-25T10:56:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:23 smithi112 ceph-mon[32799]: pgmap v59: 97 pgs: 97 active+clean; 381 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.7 MiB/s wr, 2.09k op/s 2024-04-25T10:56:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:23 smithi019 ceph-mon[29209]: pgmap v59: 97 pgs: 97 active+clean; 381 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.7 MiB/s wr, 2.09k op/s 2024-04-25T10:56:23.952 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:23.952 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-25T10:56:23.953 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:23.954 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:23.955 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:23.956 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-25T10:56:23.957 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-25T10:56:23.958 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:23.959 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-25T10:56:24.479 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:24.479 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:24.479 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:24.480 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:24.481 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:24.482 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-25T10:56:24.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:24.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-25T10:56:24.485 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:24.485 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:24.485 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:24.485 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-25T10:56:24.485 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:24.962 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-25T10:56:24.963 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:24.964 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:24.965 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:24.966 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-25T10:56:24.967 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:24.968 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:25.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-25T10:56:25.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:25.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:25.359 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:25.360 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:25.361 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:25 smithi169 ceph-mon[33049]: pgmap v60: 97 pgs: 97 active+clean; 381 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 111 MiB/s rd, 1.7 MiB/s wr, 479 op/s 2024-04-25T10:56:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:25 smithi112 ceph-mon[32799]: pgmap v60: 97 pgs: 97 active+clean; 381 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 111 MiB/s rd, 1.7 MiB/s wr, 479 op/s 2024-04-25T10:56:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:25 smithi019 ceph-mon[29209]: pgmap v60: 97 pgs: 97 active+clean; 381 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 111 MiB/s rd, 1.7 MiB/s wr, 479 op/s 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:25.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:25.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-25T10:56:25.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:25.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:25.826 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-25T10:56:26.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:26.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:26.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:26.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-25T10:56:26.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-25T10:56:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:26.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:26.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:26.443 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-25T10:56:26.845 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:26.846 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:26.847 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:26.848 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-25T10:56:26.849 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:26.850 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-25T10:56:26.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:27.232 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:27.232 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:27.232 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-25T10:56:27.232 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:27.233 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:27.234 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:27.235 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:27.236 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:27.237 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:27.238 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:27.300 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:27.301 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:27.302 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-25T10:56:27.302 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T10:56:27.302 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T10:56:27.329 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-04-25T10:56:27.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T10:56:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:27 smithi169 ceph-mon[33049]: pgmap v61: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s rd, 2.4 MiB/s wr, 764 op/s 2024-04-25T10:56:27.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:27 smithi112 ceph-mon[32799]: pgmap v61: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s rd, 2.4 MiB/s wr, 764 op/s 2024-04-25T10:56:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:27 smithi019 ceph-mon[29209]: pgmap v61: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s rd, 2.4 MiB/s wr, 764 op/s 2024-04-25T10:56:27.630 DEBUG:teuthology.parallel:result is None 2024-04-25T10:56:27.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T10:56:27.662 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T10:56:27.662 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T10:56:27.673 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:56:27.711 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T10:56:27.721 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T10:56:27.721 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T10:56:27.722 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:56:27.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T10:56:27.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T10:56:29.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:29 smithi169 ceph-mon[33049]: pgmap v62: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-04-25T10:56:29.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:29 smithi112 ceph-mon[32799]: pgmap v62: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-04-25T10:56:29.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:29 smithi019 ceph-mon[29209]: pgmap v62: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-04-25T10:56:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:31 smithi169 ceph-mon[33049]: pgmap v63: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-04-25T10:56:31.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:31 smithi112 ceph-mon[32799]: pgmap v63: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-04-25T10:56:31.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:31 smithi019 ceph-mon[29209]: pgmap v63: 97 pgs: 97 active+clean; 118 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-04-25T10:56:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:33 smithi169 ceph-mon[33049]: pgmap v64: 97 pgs: 97 active+clean; 10 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 144 MiB/s rd, 1.6 MiB/s wr, 564 op/s 2024-04-25T10:56:33.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:33 smithi112 ceph-mon[32799]: pgmap v64: 97 pgs: 97 active+clean; 10 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 144 MiB/s rd, 1.6 MiB/s wr, 564 op/s 2024-04-25T10:56:33.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:33 smithi019 ceph-mon[29209]: pgmap v64: 97 pgs: 97 active+clean; 10 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 144 MiB/s rd, 1.6 MiB/s wr, 564 op/s 2024-04-25T10:56:35.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:35 smithi169 ceph-mon[33049]: pgmap v65: 97 pgs: 97 active+clean; 10 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 902 KiB/s wr, 358 op/s 2024-04-25T10:56:35.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:35 smithi112 ceph-mon[32799]: pgmap v65: 97 pgs: 97 active+clean; 10 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 902 KiB/s wr, 358 op/s 2024-04-25T10:56:35.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:35 smithi019 ceph-mon[29209]: pgmap v65: 97 pgs: 97 active+clean; 10 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 902 KiB/s wr, 358 op/s 2024-04-25T10:56:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:37 smithi169 ceph-mon[33049]: pgmap v66: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 914 KiB/s wr, 360 op/s 2024-04-25T10:56:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:37 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:56:37.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:37 smithi112 ceph-mon[32799]: pgmap v66: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 914 KiB/s wr, 360 op/s 2024-04-25T10:56:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:37 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:56:37.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:37 smithi019 ceph-mon[29209]: pgmap v66: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 914 KiB/s wr, 360 op/s 2024-04-25T10:56:37.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:37 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: pgmap v67: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 209 KiB/s wr, 75 op/s 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:56:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:38 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: pgmap v67: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 209 KiB/s wr, 75 op/s 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:56:38.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:38 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:39.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:39.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:39.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:39.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:39.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: pgmap v67: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 209 KiB/s wr, 75 op/s 2024-04-25T10:56:39.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:56:39.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:56:39.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:38 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:56:40.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:56:40.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:56:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 -- 172.21.15.19:0/3611207378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10e070 msgr2=0x7eff7c110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 --2- 172.21.15.19:0/3611207378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10e070 0x7eff7c110540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7eff74009ca0 tx=0x7eff7402a950 comp rx=0 tx=0).stop 2024-04-25T10:56:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 -- 172.21.15.19:0/3611207378 shutdown_connections 2024-04-25T10:56:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 --2- 172.21.15.19:0/3611207378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10e070 0x7eff7c110540 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 --2- 172.21.15.19:0/3611207378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff7c10b710 0x7eff7c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 --2- 172.21.15.19:0/3611207378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7c104cb0 0x7eff7c1050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 -- 172.21.15.19:0/3611207378 >> 172.21.15.19:0/3611207378 conn(0x7eff7c078100 msgr2=0x7eff7c078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 -- 172.21.15.19:0/3611207378 shutdown_connections 2024-04-25T10:56:41.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.094+0000 7eff844d1640 1 -- 172.21.15.19:0/3611207378 wait complete. 2024-04-25T10:56:41.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 Processor -- start 2024-04-25T10:56:41.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 -- start start 2024-04-25T10:56:41.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7c104cb0 0x7eff7c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10b710 0x7eff7c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff7c10e070 0x7eff7c102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c113d00 con 0x7eff7c10b710 2024-04-25T10:56:41.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c113b80 con 0x7eff7c10e070 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff844d1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c113e80 con 0x7eff7c104cb0 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff81a6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10b710 0x7eff7c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff81a6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10b710 0x7eff7c102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55408/0 (socket says 172.21.15.19:55408) 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff81a6c640 1 -- 172.21.15.19:0/214652121 learned_addr learned my addr 172.21.15.19:0/214652121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.095+0000 7eff8226d640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7c104cb0 0x7eff7c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff82a6e640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff7c10e070 0x7eff7c102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff81a6c640 1 -- 172.21.15.19:0/214652121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7c104cb0 msgr2=0x7eff7c102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff81a6c640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7c104cb0 0x7eff7c102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff81a6c640 1 -- 172.21.15.19:0/214652121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff7c10e070 msgr2=0x7eff7c102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff81a6c640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff7c10e070 0x7eff7c102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff81a6c640 1 -- 172.21.15.19:0/214652121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c1a8a20 con 0x7eff7c10b710 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff81a6c640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10b710 0x7eff7c102ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00d950 tx=0x7eff6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff737fe640 1 -- 172.21.15.19:0/214652121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c017070 con 0x7eff7c10b710 2024-04-25T10:56:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7c1a8d10 con 0x7eff7c10b710 2024-04-25T10:56:41.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff737fe640 1 -- 172.21.15.19:0/214652121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c009c30 con 0x7eff7c10b710 2024-04-25T10:56:41.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff737fe640 1 -- 172.21.15.19:0/214652121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c00f620 con 0x7eff7c10b710 2024-04-25T10:56:41.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.096+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7c1a91f0 con 0x7eff7c10b710 2024-04-25T10:56:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 Processor -- start 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 -- start start 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e3ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e3efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff737fe640 1 -- 172.21.15.19:0/214652121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7eff6c010460 con 0x7eff7c10b710 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6e6da4d80 con 0x55e6e6a9e390 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e6e6d5e8b0 con 0x55e6e6a754d0 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.099+0000 7eff8b37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e6e6da0d40 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e3ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e3ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55410/0 (socket says 172.21.15.19:55410) 2024-04-25T10:56:41.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 -- 172.21.15.19:0/1641778185 learned_addr learned my addr 172.21.15.19:0/1641778185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff81a6c640 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff8226d640 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e3efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 -- 172.21.15.19:0/1641778185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 msgr2=0x55e6e6e4b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 -- 172.21.15.19:0/1641778185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 msgr2=0x55e6e6e3efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e3efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 -- 172.21.15.19:0/1641778185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e6d2fa40 con 0x55e6e6a9e390 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff8226d640 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e3efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff82a6e640 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e3ea60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7eff74035000 tx=0x7eff74009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a06737fd30e0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff737fe640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7eff680afc60 0x7eff680b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff70bfe640 1 -- 172.21.15.19:0/1641778185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74034040 con 0x55e6e6a9e390 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff70bfe640 1 -- 172.21.15.19:0/1641778185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff74003e20 con 0x55e6e6a9e390 2024-04-25T10:56:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.100+0000 7eff70bfe640 1 -- 172.21.15.19:0/1641778185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740042a0 con 0x55e6e6a9e390 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8226d640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7eff680afc60 0x7eff680b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff737fe640 1 -- 172.21.15.19:0/214652121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7eff6c0debe0 con 0x7eff7c10b710 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8226d640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7eff680afc60 0x7eff680b2170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55e6e6e40230 tx=0x7eff78005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8b37c740 1 -- 172.21.15.19:0/1641778185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 msgr2=0x55e6e6e3ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e3ea60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7eff74035000 tx=0x7eff74009a20 comp rx=0 tx=0).stop 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8b37c740 1 -- 172.21.15.19:0/1641778185 shutdown_connections 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e3efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1641778185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e3ea60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.101+0000 7eff8b37c740 1 -- 172.21.15.19:0/1641778185 >> 172.21.15.19:0/1641778185 conn(0x55e6e6a924e0 msgr2=0x55e6e6e3b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 -- 172.21.15.19:0/1641778185 shutdown_connections 2024-04-25T10:56:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 -- 172.21.15.19:0/1641778185 wait complete. 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 -- start start 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 Processor -- start 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e49b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6e6e4bf80 con 0x55e6e6a9e390 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e6e6d2bd10 con 0x55e6e6a754d0 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e6e6e4c100 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.102+0000 7eff8b37c740 10 client.0 _ll_register_callbacks cb 0x55e6e6d58210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55422/0 (socket says 172.21.15.19:55422) 2024-04-25T10:56:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 -- 172.21.15.19:0/1967245214 learned_addr learned my addr 172.21.15.19:0/1967245214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff8226d640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff81a6c640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e49b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 -- 172.21.15.19:0/1967245214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 msgr2=0x55e6e6e49b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e49b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 -- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 msgr2=0x55e6e6e495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 -- 172.21.15.19:0/1967245214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e6b807d0 con 0x55e6e6a9e390 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff8226d640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff82a6e640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f3d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7eff74009ca0 tx=0x7eff74003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3433ec99a915a9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff65ffb640 1 -- 172.21.15.19:0/1967245214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74053060 con 0x55e6e6a9e390 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff65ffb640 1 -- 172.21.15.19:0/1967245214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7eff74004000 con 0x55e6e6a9e390 2024-04-25T10:56:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff74003e20 con 0x55e6e6a9e390 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff65ffb640 1 -- 172.21.15.19:0/1967245214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7eff74034050 con 0x55e6e6a9e390 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.103+0000 7eff8b37c740 10 client.15828 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e6e6ab63f0 con 0x55e6e6a9e390 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff65ffb640 1 -- 172.21.15.19:0/1967245214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff74003e20 con 0x55e6e6a9e390 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff65ffb640 1 -- 172.21.15.19:0/1967245214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7eff74042070 con 0x55e6e6a9e390 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8b37c740 10 client.15828 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8b37c740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8b37c740 20 client.15828 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a5cf60 0x55e6e6e5b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8b37c740 4 client.15828 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6e6db2990 con 0x55e6e6a5cf60 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.104+0000 7eff8226d640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a5cf60 0x55e6e6e5b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.105+0000 7eff65ffb640 10 client.15828 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.105+0000 7eff8226d640 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a5cf60 0x55e6e6e5b4c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c70f5d886774618 server_cookie=eb40661074dbb5b7 in_seq=0 out_seq=0 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.105+0000 7eff65ffb640 1 -- 172.21.15.19:0/1967245214 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6e6db2990 con 0x55e6e6a5cf60 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff65ffb640 10 client.15828 handle_command_reply: tid=0 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 client.15828 shutdown 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 msgr2=0x55e6e6e4f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f3d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7eff74009ca0 tx=0x7eff74003c20 comp rx=0 tx=0).stop 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 shutdown_connections 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a5cf60 0x55e6e6e5b4c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f3d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e49b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff65ffb640 0 client.15828 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 --2- 172.21.15.19:0/1967245214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 >> 172.21.15.19:0/1967245214 conn(0x55e6e6a924e0 msgr2=0x55e6e6e40780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 shutdown_connections 2024-04-25T10:56:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 1 -- 172.21.15.19:0/1967245214 wait complete. 2024-04-25T10:56:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.106+0000 7eff8b37c740 20 client.15828 trim_cache size 0 max 16384 2024-04-25T10:56:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.124+0000 7eff8b37c740 1 Processor -- start 2024-04-25T10:56:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.124+0000 7eff8b37c740 1 -- start start 2024-04-25T10:56:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e4b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff8b37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1afae0 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff8b37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1b2360 con 0x55e6e6a754d0 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff8b37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1b0150 con 0x55e6e6a9e390 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55424/0 (socket says 172.21.15.19:55424) 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 -- 172.21.15.19:0/2587833484 learned_addr learned my addr 172.21.15.19:0/2587833484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff8226d640 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e4b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff82a6e640 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 -- 172.21.15.19:0/2587833484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a9e390 msgr2=0x55e6e6e4f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 -- 172.21.15.19:0/2587833484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 msgr2=0x55e6e6e4b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e4b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 -- 172.21.15.19:0/2587833484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e6a760f0 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff81a6c640 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4f3c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0a2e30 tx=0x7eff6c0e2350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5708a7979644582a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff67fff640 1 -- 172.21.15.19:0/2587833484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c020040 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff67fff640 1 -- 172.21.15.19:0/2587833484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c0a68c0 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.125+0000 7eff67fff640 1 -- 172.21.15.19:0/2587833484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0a6db0 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 -- 172.21.15.19:0/2587833484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a7d6b0 msgr2=0x55e6e6e4f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4f3c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0a2e30 tx=0x7eff6c0e2350 comp rx=0 tx=0).stop 2024-04-25T10:56:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 -- 172.21.15.19:0/2587833484 shutdown_connections 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a9e390 0x55e6e6e4f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e4f3c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2587833484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a754d0 0x55e6e6e4b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 -- 172.21.15.19:0/2587833484 >> 172.21.15.19:0/2587833484 conn(0x55e6e6a924e0 msgr2=0x55e6e6e48340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 -- 172.21.15.19:0/2587833484 shutdown_connections 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.126+0000 7eff8b37c740 1 -- 172.21.15.19:0/2587833484 wait complete. 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 -- start start 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 Processor -- start 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a754d0 0x55e6e6e529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e52f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a9e390 0x55e6e6ec15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1b0820 con 0x55e6e6a754d0 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1b1c90 con 0x55e6e6a9e390 2024-04-25T10:56:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1b0ef0 con 0x55e6e6a7d6b0 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8b37c740 10 client.0 _ll_register_callbacks cb 0x7eff6c0e2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a754d0 0x55e6e6e529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a754d0 0x55e6e6e529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55436/0 (socket says 172.21.15.19:55436) 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 -- 172.21.15.19:0/2524749526 learned_addr learned my addr 172.21.15.19:0/2524749526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff81a6c640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e52f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 -- 172.21.15.19:0/2524749526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 msgr2=0x55e6e6e52f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff82a6e640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a9e390 0x55e6e6ec15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e52f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 -- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a9e390 msgr2=0x55e6e6ec15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a9e390 0x55e6e6ec15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 -- 172.21.15.19:0/2524749526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e6d654b0 con 0x55e6e6a754d0 2024-04-25T10:56:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff81a6c640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e52f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.127+0000 7eff8226d640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a754d0 0x55e6e6e529f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55e6e6e4c100 tx=0x7eff780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b87fc17eb7b4e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff65ffb640 1 -- 172.21.15.19:0/2524749526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7802a480 con 0x55e6e6a754d0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff65ffb640 1 -- 172.21.15.19:0/2524749526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7eff7802a620 con 0x55e6e6a754d0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6c0a68c0 con 0x55e6e6a754d0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff65ffb640 1 -- 172.21.15.19:0/2524749526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7eff7802a820 con 0x55e6e6a754d0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 10 client.15834 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e6e6ec2ae0 con 0x55e6e6a754d0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff65ffb640 1 -- 172.21.15.19:0/2524749526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c0a68c0 con 0x55e6e6a754d0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff65ffb640 1 -- 172.21.15.19:0/2524749526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7eff7402f520 con 0x55e6e6a754d0 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 10 client.15834 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 20 client.15834 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a323a0 0x55e6e6ec5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 4 client.15834 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.128+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6e6e3f330 con 0x55e6e6a323a0 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff81a6c640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a323a0 0x55e6e6ec5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff65ffb640 10 client.15834 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff81a6c640 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a323a0 0x55e6e6ec5250 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b76f14b25db61c72 server_cookie=99a930f27ef6a6c2 in_seq=0 out_seq=0 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff65ffb640 1 -- 172.21.15.19:0/2524749526 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e6e6e3f330 con 0x55e6e6a323a0 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff65ffb640 10 client.15834 handle_command_reply: tid=0 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 client.15834 shutdown 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a754d0 msgr2=0x55e6e6e529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a754d0 0x55e6e6e529f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55e6e6e4c100 tx=0x7eff780046c0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 shutdown_connections 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e6e6a323a0 0x55e6e6ec5250 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6e6a9e390 0x55e6e6ec15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6e6a7d6b0 0x55e6e6e52f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff65ffb640 0 client.15834 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 --2- 172.21.15.19:0/2524749526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6e6a754d0 0x55e6e6e529f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 >> 172.21.15.19:0/2524749526 conn(0x55e6e6a924e0 msgr2=0x55e6e6e40660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.129+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 shutdown_connections 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff8b37c740 1 -- 172.21.15.19:0/2524749526 wait complete. 2024-04-25T10:56:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff8b37c740 20 client.15834 trim_cache size 0 max 16384 2024-04-25T10:56:41.138 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:56:41.138 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:56:41.138 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:56:41.138 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7eff680afc60 msgr2=0x7eff680b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7eff680afc60 0x7eff680b2170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55e6e6e40230 tx=0x7eff78005f90 comp rx=0 tx=0).stop 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10b710 msgr2=0x7eff7c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10b710 0x7eff7c102ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00d950 tx=0x7eff6c00de10 comp rx=0 tx=0).stop 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 shutdown_connections 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7eff680afc60 0x7eff680b2170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff7c10e070 0x7eff7c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff7c10b710 0x7eff7c102ab0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 --2- 172.21.15.19:0/214652121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7c104cb0 0x7eff7c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 >> 172.21.15.19:0/214652121 conn(0x7eff7c078100 msgr2=0x7eff7c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 shutdown_connections 2024-04-25T10:56:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.130+0000 7eff844d1640 1 -- 172.21.15.19:0/214652121 wait complete. 2024-04-25T10:56:41.142 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:56:41.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 -- 172.21.15.19:0/570432820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5010d380 msgr2=0x7ffb5010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 --2- 172.21.15.19:0/570432820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5010d380 0x7ffb5010f7a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009ca0 tx=0x7ffb4802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 -- 172.21.15.19:0/570432820 shutdown_connections 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 --2- 172.21.15.19:0/570432820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb5010d380 0x7ffb5010f7a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 --2- 172.21.15.19:0/570432820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb50102af0 0x7ffb50102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 --2- 172.21.15.19:0/570432820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb50108a20 0x7ffb501025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 -- 172.21.15.19:0/570432820 >> 172.21.15.19:0/570432820 conn(0x7ffb50078110 msgr2=0x7ffb50078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 -- 172.21.15.19:0/570432820 shutdown_connections 2024-04-25T10:56:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 -- 172.21.15.19:0/570432820 wait complete. 2024-04-25T10:56:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.254+0000 7ffb55642640 1 Processor -- start 2024-04-25T10:56:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb55642640 1 -- start start 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb55642640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb50102af0 0x7ffb50111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb55642640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb50108a20 0x7ffb501122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb55642640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5010d380 0x7ffb50112830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb55642640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5006adb0 con 0x7ffb50102af0 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb55642640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5006ac30 con 0x7ffb5010d380 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb55642640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb50113ef0 con 0x7ffb50108a20 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb50102af0 0x7ffb50111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb50102af0 0x7ffb50111db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55456/0 (socket says 172.21.15.19:55456) 2024-04-25T10:56:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 -- 172.21.15.19:0/2653239140 learned_addr learned my addr 172.21.15.19:0/2653239140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4f7fe640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5010d380 0x7ffb50112830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4e7fc640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb50108a20 0x7ffb501122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 -- 172.21.15.19:0/2653239140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb50108a20 msgr2=0x7ffb501122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb50108a20 0x7ffb501122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 -- 172.21.15.19:0/2653239140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5010d380 msgr2=0x7ffb50112830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5010d380 0x7ffb50112830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4effd640 1 -- 172.21.15.19:0/2653239140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb501a8a60 con 0x7ffb50102af0 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.255+0000 7ffb4e7fc640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb50108a20 0x7ffb501122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.256+0000 7ffb4effd640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb50102af0 0x7ffb50111db0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000d770 tx=0x7ffb4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.256+0000 7ffb37fff640 1 -- 172.21.15.19:0/2653239140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40017070 con 0x7ffb50102af0 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.256+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb501a8d50 con 0x7ffb50102af0 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.256+0000 7ffb37fff640 1 -- 172.21.15.19:0/2653239140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb40009c30 con 0x7ffb50102af0 2024-04-25T10:56:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.256+0000 7ffb37fff640 1 -- 172.21.15.19:0/2653239140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4000f620 con 0x7ffb50102af0 2024-04-25T10:56:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.256+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb501a9200 con 0x7ffb50102af0 2024-04-25T10:56:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.257+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb50103cd0 con 0x7ffb50102af0 2024-04-25T10:56:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.259+0000 7ffb37fff640 1 -- 172.21.15.19:0/2653239140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ffb4001a070 con 0x7ffb50102af0 2024-04-25T10:56:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.259+0000 7ffb37fff640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb380afc60 0x7ffb380b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.259+0000 7ffb37fff640 1 -- 172.21.15.19:0/2653239140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ffb400deb10 con 0x7ffb50102af0 2024-04-25T10:56:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.259+0000 7ffb4e7fc640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb380afc60 0x7ffb380b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.260+0000 7ffb4e7fc640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb380afc60 0x7ffb380b2170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffb5010e960 tx=0x7ffb44005e70 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.263+0000 7ffb37fff640 1 -- 172.21.15.19:0/2653239140 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb40009de0 con 0x7ffb50102af0 2024-04-25T10:56:41.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:41 smithi112 ceph-mon[32799]: pgmap v68: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 209 KiB/s wr, 75 op/s 2024-04-25T10:56:41.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:41 smithi019 ceph-mon[29209]: pgmap v68: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 209 KiB/s wr, 75 op/s 2024-04-25T10:56:41.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.488+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb5006afa0 con 0x7ffb50102af0 2024-04-25T10:56:41.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.489+0000 7ffb37fff640 1 -- 172.21.15.19:0/2653239140 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7ffb400a7380 con 0x7ffb50102af0 2024-04-25T10:56:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:56:41.490 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:56:41.490 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb380afc60 msgr2=0x7ffb380b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb380afc60 0x7ffb380b2170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffb5010e960 tx=0x7ffb44005e70 comp rx=0 tx=0).stop 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb50102af0 msgr2=0x7ffb50111db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb50102af0 0x7ffb50111db0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000d770 tx=0x7ffb4000dc30 comp rx=0 tx=0).stop 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 shutdown_connections 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb380afc60 0x7ffb380b2170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5010d380 0x7ffb50112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb50108a20 0x7ffb501122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 --2- 172.21.15.19:0/2653239140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb50102af0 0x7ffb50111db0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 >> 172.21.15.19:0/2653239140 conn(0x7ffb50078110 msgr2=0x7ffb501040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 shutdown_connections 2024-04-25T10:56:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.491+0000 7ffb55642640 1 -- 172.21.15.19:0/2653239140 wait complete. 2024-04-25T10:56:41.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:41 smithi169 ceph-mon[33049]: pgmap v68: 97 pgs: 97 active+clean; 11 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 209 KiB/s wr, 75 op/s 2024-04-25T10:56:41.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 -- 172.21.15.19:0/1298478355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c10e570 msgr2=0x7fe26c110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 --2- 172.21.15.19:0/1298478355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c10e570 0x7fe26c110990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe264009ca0 tx=0x7fe26402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 -- 172.21.15.19:0/1298478355 shutdown_connections 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 --2- 172.21.15.19:0/1298478355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c10e570 0x7fe26c110990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 --2- 172.21.15.19:0/1298478355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26c102070 0x7fe26c102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 --2- 172.21.15.19:0/1298478355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe26c109bc0 0x7fe26c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 -- 172.21.15.19:0/1298478355 >> 172.21.15.19:0/1298478355 conn(0x7fe26c077920 msgr2=0x7fe26c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 -- 172.21.15.19:0/1298478355 shutdown_connections 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.616+0000 7fe273030640 1 -- 172.21.15.19:0/1298478355 wait complete. 2024-04-25T10:56:41.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 Processor -- start 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 -- start start 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c102070 0x7fe26c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe26c109bc0 0x7fe26c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26c10e570 0x7fe26c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c113d30 con 0x7fe26c102070 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c113bb0 con 0x7fe26c10e570 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe273030640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c113eb0 con 0x7fe26c109bc0 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe270dcc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c102070 0x7fe26c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe270dcc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c102070 0x7fe26c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55478/0 (socket says 172.21.15.19:55478) 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe270dcc640 1 -- 172.21.15.19:0/563096557 learned_addr learned my addr 172.21.15.19:0/563096557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.617+0000 7fe263fff640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe26c109bc0 0x7fe26c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe270dcc640 1 -- 172.21.15.19:0/563096557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe26c109bc0 msgr2=0x7fe26c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe2715cd640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26c10e570 0x7fe26c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe270dcc640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe26c109bc0 0x7fe26c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe270dcc640 1 -- 172.21.15.19:0/563096557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26c10e570 msgr2=0x7fe26c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe270dcc640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26c10e570 0x7fe26c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe270dcc640 1 -- 172.21.15.19:0/563096557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c19f550 con 0x7fe26c102070 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe270dcc640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c102070 0x7fe26c19a6f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe268005d00 tx=0x7fe26800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe261ffb640 1 -- 172.21.15.19:0/563096557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268017070 con 0x7fe26c102070 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c19f840 con 0x7fe26c102070 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe261ffb640 1 -- 172.21.15.19:0/563096557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe268009c30 con 0x7fe26c102070 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe261ffb640 1 -- 172.21.15.19:0/563096557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26800f620 con 0x7fe26c102070 2024-04-25T10:56:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.618+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c19fd40 con 0x7fe26c102070 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 Processor -- start 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe261ffb640 1 -- 172.21.15.19:0/563096557 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe26801a070 con 0x7fe26c102070 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 -- start start 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a2932390 0x55f9a2cd3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29094d0 0x55f9a2cd3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29116b0 0x55f9a2ce06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f9a2c38d80 con 0x55f9a2932390 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9a2bf2a20 con 0x55f9a29116b0 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe279fb2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f9a2c34d40 con 0x55f9a29094d0 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe2715cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a2932390 0x55f9a2cd3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe2715cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a2932390 0x55f9a2cd3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55480/0 (socket says 172.21.15.19:55480) 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe2715cd640 1 -- 172.21.15.19:0/1993189385 learned_addr learned my addr 172.21.15.19:0/1993189385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.621+0000 7fe270dcc640 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29094d0 0x55f9a2cd3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe263fff640 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29116b0 0x55f9a2ce06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe2715cd640 1 -- 172.21.15.19:0/1993189385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29094d0 msgr2=0x55f9a2cd3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe2715cd640 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29094d0 0x55f9a2cd3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe2715cd640 1 -- 172.21.15.19:0/1993189385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29116b0 msgr2=0x55f9a2ce06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe2715cd640 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29116b0 0x55f9a2ce06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe261ffb640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2540afc60 0x7fe2540b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe2715cd640 1 -- 172.21.15.19:0/1993189385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a2bfdc30 con 0x55f9a2932390 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe263fff640 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29116b0 0x55f9a2ce06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe270dcc640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2540afc60 0x7fe2540b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe2715cd640 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a2932390 0x55f9a2cd3810 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe264005cc0 tx=0x7fe264009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444b9b68ee62b25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe261ffb640 1 -- 172.21.15.19:0/563096557 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe2680deaa0 con 0x7fe26c102070 2024-04-25T10:56:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe252ffd640 1 -- 172.21.15.19:0/1993189385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264031cf0 con 0x55f9a2932390 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe252ffd640 1 -- 172.21.15.19:0/1993189385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe264036070 con 0x55f9a2932390 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.622+0000 7fe252ffd640 1 -- 172.21.15.19:0/1993189385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2640042a0 con 0x55f9a2932390 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 -- 172.21.15.19:0/1993189385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a2932390 msgr2=0x55f9a2cd3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe270dcc640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2540afc60 0x7fe2540b2170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2680dc2d0 tx=0x7fe2680e25f0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a2932390 0x55f9a2cd3810 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe264005cc0 tx=0x7fe264009800 comp rx=0 tx=0).stop 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 -- 172.21.15.19:0/1993189385 shutdown_connections 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29116b0 0x55f9a2ce06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29094d0 0x55f9a2cd3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1993189385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a2932390 0x55f9a2cd3810 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 -- 172.21.15.19:0/1993189385 >> 172.21.15.19:0/1993189385 conn(0x55f9a29264e0 msgr2=0x55f9a2ccffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 -- 172.21.15.19:0/1993189385 shutdown_connections 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.623+0000 7fe279fb2740 1 -- 172.21.15.19:0/1993189385 wait complete. 2024-04-25T10:56:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 -- start start 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 Processor -- start 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2cde230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2cde770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a2932390 0x55f9a2cea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f9a2ce0d30 con 0x55f9a29116b0 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9a2bbfdc0 con 0x55f9a29094d0 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f9a2ce0eb0 con 0x55f9a2932390 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe279fb2740 10 client.0 _ll_register_callbacks cb 0x55f9a2bdb600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe263fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2cde770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe263fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2cde770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55486/0 (socket says 172.21.15.19:55486) 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe263fff640 1 -- 172.21.15.19:0/4064310173 learned_addr learned my addr 172.21.15.19:0/4064310173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe2715cd640 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a2932390 0x55f9a2cea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.624+0000 7fe2715cd640 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2cde230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe263fff640 1 -- 172.21.15.19:0/4064310173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a2932390 msgr2=0x55f9a2cea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe263fff640 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a2932390 0x55f9a2cea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe263fff640 1 -- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 msgr2=0x55f9a2cde230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe263fff640 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2cde230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe263fff640 1 -- 172.21.15.19:0/4064310173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a294a3f0 con 0x55f9a29116b0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe263fff640 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2cde770 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe25c005bc0 tx=0x7fe25c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab2bfe5efe0f72e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe24bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe24bfff640 1 -- 172.21.15.19:0/4064310173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c01b440 con 0x55f9a29116b0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe24bfff640 1 -- 172.21.15.19:0/4064310173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe25c01b5e0 con 0x55f9a29116b0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe264036070 con 0x55f9a29116b0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe24bfff640 1 -- 172.21.15.19:0/4064310173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe25c01b7e0 con 0x55f9a29116b0 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe279fb2740 10 client.15852 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:41.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9a2a147d0 con 0x55f9a29116b0 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.625+0000 7fe24bfff640 1 -- 172.21.15.19:0/4064310173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe264036070 con 0x55f9a29116b0 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe24bfff640 1 -- 172.21.15.19:0/4064310173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe25c019020 con 0x55f9a29116b0 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe279fb2740 10 client.15852 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe279fb2740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe279fb2740 20 client.15852 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe279fb2740 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a28f0f60 0x55f9a2cf0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe279fb2740 4 client.15852 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a2c46990 con 0x55f9a28f0f60 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe2715cd640 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a28f0f60 0x55f9a2cf0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe2715cd640 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a28f0f60 0x55f9a2cf0660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcce4784223d4f41 server_cookie=a1452c9e03d67249 in_seq=0 out_seq=0 2024-04-25T10:56:41.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.626+0000 7fe24bfff640 10 client.15852 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe24bfff640 1 -- 172.21.15.19:0/4064310173 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9a2c46990 con 0x55f9a28f0f60 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe24bfff640 10 client.15852 handle_command_reply: tid=0 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 client.15852 shutdown 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 msgr2=0x55f9a2cde770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2cde770 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe25c005bc0 tx=0x7fe25c00bf00 comp rx=0 tx=0).stop 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 shutdown_connections 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a28f0f60 0x55f9a2cf0660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a2932390 0x55f9a2cea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2cde770 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe24bfff640 0 client.15852 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 --2- 172.21.15.19:0/4064310173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2cde230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 >> 172.21.15.19:0/4064310173 conn(0x55f9a29264e0 msgr2=0x55f9a2cd5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 shutdown_connections 2024-04-25T10:56:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.627+0000 7fe279fb2740 1 -- 172.21.15.19:0/4064310173 wait complete. 2024-04-25T10:56:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.628+0000 7fe279fb2740 20 client.15852 trim_cache size 0 max 16384 2024-04-25T10:56:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.641+0000 7fe279fb2740 1 Processor -- start 2024-04-25T10:56:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.641+0000 7fe279fb2740 1 -- start start 2024-04-25T10:56:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29094d0 0x55f9a2cf5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29116b0 0x55f9a2cf6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe279fb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1ad8b0 con 0x55f9a29094d0 2024-04-25T10:56:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe279fb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1b0130 con 0x55f9a28f0f60 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe279fb2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1adf20 con 0x55f9a29116b0 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29094d0 0x55f9a2cf5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29094d0 0x55f9a2cf5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55492/0 (socket says 172.21.15.19:55492) 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 -- 172.21.15.19:0/165867882 learned_addr learned my addr 172.21.15.19:0/165867882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe2715cd640 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe2715cd640 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29116b0 0x55f9a2cf6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 -- 172.21.15.19:0/165867882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29116b0 msgr2=0x55f9a2cf6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29116b0 0x55f9a2cf6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 -- 172.21.15.19:0/165867882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a28f0f60 msgr2=0x55f9a2ce9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 -- 172.21.15.19:0/165867882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a2ccb500 con 0x55f9a29094d0 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.642+0000 7fe263fff640 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29094d0 0x55f9a2cf5e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe25c001eb0 tx=0x7fe25c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93704c44157f3a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe2527fc640 1 -- 172.21.15.19:0/165867882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c028040 con 0x55f9a29094d0 2024-04-25T10:56:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe2527fc640 1 -- 172.21.15.19:0/165867882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25c01b980 con 0x55f9a29094d0 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe2527fc640 1 -- 172.21.15.19:0/165867882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0052a0 con 0x55f9a29094d0 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 -- 172.21.15.19:0/165867882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29094d0 msgr2=0x55f9a2cf5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29094d0 0x55f9a2cf5e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe25c001eb0 tx=0x7fe25c003f70 comp rx=0 tx=0).stop 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 -- 172.21.15.19:0/165867882 shutdown_connections 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a29116b0 0x55f9a2cf6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29094d0 0x55f9a2cf5e20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 --2- 172.21.15.19:0/165867882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 -- 172.21.15.19:0/165867882 >> 172.21.15.19:0/165867882 conn(0x55f9a29264e0 msgr2=0x55f9a2cdc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 -- 172.21.15.19:0/165867882 shutdown_connections 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.643+0000 7fe279fb2740 1 -- 172.21.15.19:0/165867882 wait complete. 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 -- start start 2024-04-25T10:56:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 Processor -- start 2024-04-25T10:56:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2ce17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2ce1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1ae5f0 con 0x55f9a29116b0 2024-04-25T10:56:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1afa60 con 0x55f9a29094d0 2024-04-25T10:56:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1aecc0 con 0x55f9a28f0f60 2024-04-25T10:56:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe279fb2740 10 client.0 _ll_register_callbacks cb 0x7fe25c005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe2715cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2ce1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe2715cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2ce1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55498/0 (socket says 172.21.15.19:55498) 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe2715cd640 1 -- 172.21.15.19:0/1856208701 learned_addr learned my addr 172.21.15.19:0/1856208701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe2715cd640 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.644+0000 7fe263fff640 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2ce17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe2715cd640 1 -- 172.21.15.19:0/1856208701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a28f0f60 msgr2=0x55f9a2ce64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe2715cd640 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe2715cd640 1 -- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 msgr2=0x55f9a2ce17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe2715cd640 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2ce17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe2715cd640 1 -- 172.21.15.19:0/1856208701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a2ce2270 con 0x55f9a29116b0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe2715cd640 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2ce1d30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe26402c010 tx=0x7fe264005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5587bd68af2ad9af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe24bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe24bfff640 1 -- 172.21.15.19:0/1856208701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264049040 con 0x55f9a29116b0 2024-04-25T10:56:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe24bfff640 1 -- 172.21.15.19:0/1856208701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe264031450 con 0x55f9a29116b0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe25c01b980 con 0x55f9a29116b0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe24bfff640 1 -- 172.21.15.19:0/1856208701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe264031650 con 0x55f9a29116b0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe279fb2740 10 client.15858 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9a2d58640 con 0x55f9a29116b0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.645+0000 7fe24bfff640 1 -- 172.21.15.19:0/1856208701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25c01b980 con 0x55f9a29116b0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe24bfff640 1 -- 172.21.15.19:0/1856208701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe26404f040 con 0x55f9a29116b0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe279fb2740 10 client.15858 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe279fb2740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe279fb2740 20 client.15858 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a2932390 0x55f9a2d5a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe279fb2740 4 client.15858 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9a2c40800 con 0x55f9a2932390 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe263fff640 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a2932390 0x55f9a2d5a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe263fff640 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a2932390 0x55f9a2d5a930 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5f0d092329c65 server_cookie=f48f97842c0d604 in_seq=0 out_seq=0 2024-04-25T10:56:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.646+0000 7fe24bfff640 10 client.15858 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe24bfff640 1 -- 172.21.15.19:0/1856208701 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9a2c40800 con 0x55f9a2932390 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe24bfff640 10 client.15858 handle_command_reply: tid=0 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 client.15858 shutdown 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 msgr2=0x55f9a2ce1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2ce1d30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe26402c010 tx=0x7fe264005c50 comp rx=0 tx=0).stop 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 shutdown_connections 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9a2932390 0x55f9a2d5a930 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a29116b0 0x55f9a2ce1d30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9a29094d0 0x55f9a2ce17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 --2- 172.21.15.19:0/1856208701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9a28f0f60 0x55f9a2ce64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe24bfff640 0 client.15858 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 >> 172.21.15.19:0/1856208701 conn(0x55f9a29264e0 msgr2=0x55f9a2cdc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 shutdown_connections 2024-04-25T10:56:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 1 -- 172.21.15.19:0/1856208701 wait complete. 2024-04-25T10:56:41.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.647+0000 7fe279fb2740 20 client.15858 trim_cache size 0 max 16384 2024-04-25T10:56:41.657 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2540afc60 msgr2=0x7fe2540b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2540afc60 0x7fe2540b2170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2680dc2d0 tx=0x7fe2680e25f0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c102070 msgr2=0x7fe26c19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c102070 0x7fe26c19a6f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe268005d00 tx=0x7fe26800dde0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 shutdown_connections 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2540afc60 0x7fe2540b2170 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26c10e570 0x7fe26c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe26c109bc0 0x7fe26c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 --2- 172.21.15.19:0/563096557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26c102070 0x7fe26c19a6f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:41.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 >> 172.21.15.19:0/563096557 conn(0x7fe26c077920 msgr2=0x7fe26c10a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 shutdown_connections 2024-04-25T10:56:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:41.648+0000 7fe273030640 1 -- 172.21.15.19:0/563096557 wait complete. 2024-04-25T10:56:42.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:42 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2653239140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:56:42.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:42 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2653239140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:56:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:42 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2653239140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:56:42.660 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:56:42.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:56:42.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:56:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 -- 172.21.15.19:0/3583253831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 msgr2=0x7f4288102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 --2- 172.21.15.19:0/3583253831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f4288102b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f427c009900 tx=0x7f427c02a410 comp rx=0 tx=0).stop 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 -- 172.21.15.19:0/3583253831 shutdown_connections 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 --2- 172.21.15.19:0/3583253831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42881031c0 0x7f428810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 --2- 172.21.15.19:0/3583253831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f4288102b50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 --2- 172.21.15.19:0/3583253831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288108a10 0x7f4288102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 -- 172.21.15.19:0/3583253831 >> 172.21.15.19:0/3583253831 conn(0x7f4288078110 msgr2=0x7f4288078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 -- 172.21.15.19:0/3583253831 shutdown_connections 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.792+0000 7f428dff3640 1 -- 172.21.15.19:0/3583253831 wait complete. 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 Processor -- start 2024-04-25T10:56:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 -- start start 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42881031c0 0x7f428819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288108a10 0x7f428819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f428819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4288112c30 con 0x7f4288108eb0 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4288112ab0 con 0x7f42881031c0 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f428dff3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4288112db0 con 0x7f4288108a10 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f4287fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f428819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f4287fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f428819c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50870/0 (socket says 172.21.15.19:50870) 2024-04-25T10:56:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f4287fff640 1 -- 172.21.15.19:0/360097794 learned_addr learned my addr 172.21.15.19:0/360097794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f42877fe640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42881031c0 0x7f428819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.793+0000 7f4286ffd640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288108a10 0x7f428819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4287fff640 1 -- 172.21.15.19:0/360097794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288108a10 msgr2=0x7f428819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4287fff640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288108a10 0x7f428819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4287fff640 1 -- 172.21.15.19:0/360097794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42881031c0 msgr2=0x7f428819b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4287fff640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42881031c0 0x7f428819b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4287fff640 1 -- 172.21.15.19:0/360097794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42881a68f0 con 0x7f4288108eb0 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4287fff640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f428819c050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f428000eb10 tx=0x7f428000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4284ff9640 1 -- 172.21.15.19:0/360097794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280004b20 con 0x7f4288108eb0 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42881a6be0 con 0x7f4288108eb0 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4284ff9640 1 -- 172.21.15.19:0/360097794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4280004cc0 con 0x7f4288108eb0 2024-04-25T10:56:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f4284ff9640 1 -- 172.21.15.19:0/360097794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280010670 con 0x7f4288108eb0 2024-04-25T10:56:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.794+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42881a7100 con 0x7f4288108eb0 2024-04-25T10:56:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 Processor -- start 2024-04-25T10:56:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4284ff9640 1 -- 172.21.15.19:0/360097794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4280005020 con 0x7f4288108eb0 2024-04-25T10:56:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 -- start start 2024-04-25T10:56:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b118219b00 con 0x55b117ef2730 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1181b5920 con 0x55b117f13410 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4294f81740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b118215ac0 con 0x55b117eea550 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.797+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50872/0 (socket says 172.21.15.19:50872) 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 -- 172.21.15.19:0/1973216103 learned_addr learned my addr 172.21.15.19:0/1973216103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f42877fe640 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4284ff9640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42700afc60 0x7f42700b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4287fff640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42700afc60 0x7f42700b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 -- 172.21.15.19:0/1973216103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 msgr2=0x55b1182b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 -- 172.21.15.19:0/1973216103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 msgr2=0x55b1182b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 -- 172.21.15.19:0/1973216103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b117ff37e0 con 0x55b117ef2730 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4284ff9640 1 -- 172.21.15.19:0/360097794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f42800df5f0 con 0x7f4288108eb0 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4286ffd640 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182b40a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55b1182b0790 tx=0x7f427c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99b480f7cc57e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f426dffb640 1 -- 172.21.15.19:0/1973216103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c044020 con 0x55b117ef2730 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f426dffb640 1 -- 172.21.15.19:0/1973216103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c0316a0 con 0x55b117ef2730 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f426dffb640 1 -- 172.21.15.19:0/1973216103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c035070 con 0x55b117ef2730 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.798+0000 7f4287fff640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42700afc60 0x7f42700b2170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42800dd370 tx=0x7f42800ca040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 -- 172.21.15.19:0/1973216103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 msgr2=0x55b1182b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182b40a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55b1182b0790 tx=0x7f427c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 -- 172.21.15.19:0/1973216103 shutdown_connections 2024-04-25T10:56:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182b40a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 --2- 172.21.15.19:0/1973216103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 -- 172.21.15.19:0/1973216103 >> 172.21.15.19:0/1973216103 conn(0x55b117f07560 msgr2=0x55b1182afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 -- 172.21.15.19:0/1973216103 shutdown_connections 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.799+0000 7f4294f81740 1 -- 172.21.15.19:0/1973216103 wait complete. 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 -- start start 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 Processor -- start 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117eea550 0x55b1182bdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117ef2730 0x55b1182be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117f13410 0x55b1182c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1182c0910 con 0x55b117eea550 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1181b35d0 con 0x55b117ef2730 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b1182c0a90 con 0x55b117f13410 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4294f81740 10 client.0 _ll_register_callbacks cb 0x55b1181a1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.800+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117eea550 0x55b1182bdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117eea550 0x55b1182bdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50876/0 (socket says 172.21.15.19:50876) 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 -- 172.21.15.19:0/3772124235 learned_addr learned my addr 172.21.15.19:0/3772124235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f42877fe640 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117ef2730 0x55b1182be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 -- 172.21.15.19:0/3772124235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117f13410 msgr2=0x55b1182c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117f13410 0x55b1182c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 -- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117ef2730 msgr2=0x55b1182be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117ef2730 0x55b1182be150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 -- 172.21.15.19:0/3772124235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b118053f70 con 0x55b117eea550 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f42877fe640 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117ef2730 0x55b1182be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4286ffd640 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117eea550 0x55b1182bdc10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f427c031a80 tx=0x7f427c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72f844dc472f1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4266ffd640 1 -- 172.21.15.19:0/3772124235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c04d070 con 0x55b117eea550 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427c0316a0 con 0x55b117eea550 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4266ffd640 1 -- 172.21.15.19:0/3772124235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f427c0055b0 con 0x55b117eea550 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4266ffd640 1 -- 172.21.15.19:0/3772124235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f427c044050 con 0x55b117eea550 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4294f81740 10 client.15870 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.801+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b117eeb170 con 0x55b117eea550 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4266ffd640 1 -- 172.21.15.19:0/3772124235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c0316a0 con 0x55b117eea550 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4266ffd640 1 -- 172.21.15.19:0/3772124235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f427c003cd0 con 0x55b117eea550 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4294f81740 10 client.15870 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4294f81740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4294f81740 20 client.15870 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4294f81740 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ed21f0 0x55b1182cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4294f81740 4 client.15870 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b118227710 con 0x55b117ed21f0 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.802+0000 7f42877fe640 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ed21f0 0x55b1182cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.803+0000 7f4266ffd640 10 client.15870 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.803+0000 7f42877fe640 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ed21f0 0x55b1182cfe10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5446d7da1cf0500f server_cookie=d289737131d2a0c in_seq=0 out_seq=0 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.803+0000 7f4266ffd640 1 -- 172.21.15.19:0/3772124235 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b118227710 con 0x55b117ed21f0 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.803+0000 7f4266ffd640 10 client.15870 handle_command_reply: tid=0 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 client.15870 shutdown 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117eea550 msgr2=0x55b1182bdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117eea550 0x55b1182bdc10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f427c031a80 tx=0x7f427c0095d0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 shutdown_connections 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ed21f0 0x55b1182cfe10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117f13410 0x55b1182c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4266ffd640 0 client.15870 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117ef2730 0x55b1182be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 --2- 172.21.15.19:0/3772124235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117eea550 0x55b1182bdc10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 >> 172.21.15.19:0/3772124235 conn(0x55b117f07560 msgr2=0x55b1182b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 shutdown_connections 2024-04-25T10:56:42.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 1 -- 172.21.15.19:0/3772124235 wait complete. 2024-04-25T10:56:42.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.804+0000 7f4294f81740 20 client.15870 trim_cache size 0 max 16384 2024-04-25T10:56:42.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 Processor -- start 2024-04-25T10:56:42.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 -- start start 2024-04-25T10:56:42.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117eea550 0x55b1182aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117ef2730 0x55b1182b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117f13410 0x55b1182b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42881ad910 con 0x55b117f13410 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42881b0190 con 0x55b117eea550 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4294f81740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42881adf80 con 0x55b117ef2730 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117f13410 0x55b1182b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117f13410 0x55b1182b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50880/0 (socket says 172.21.15.19:50880) 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 -- 172.21.15.19:0/4132718345 learned_addr learned my addr 172.21.15.19:0/4132718345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 -- 172.21.15.19:0/4132718345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117ef2730 msgr2=0x55b1182b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f42877fe640 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117ef2730 0x55b1182b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117ef2730 0x55b1182b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 -- 172.21.15.19:0/4132718345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117eea550 msgr2=0x55b1182aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117eea550 0x55b1182aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f4286ffd640 1 -- 172.21.15.19:0/4132718345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1182c0a70 con 0x55b117f13410 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.823+0000 7f42877fe640 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117ef2730 0x55b1182b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4286ffd640 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117f13410 0x55b1182b78c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f427c02a930 tx=0x7f427c005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651cfcb58fb39ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f426d7fa640 1 -- 172.21.15.19:0/4132718345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c04d040 con 0x55b117f13410 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f426d7fa640 1 -- 172.21.15.19:0/4132718345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c0314a0 con 0x55b117f13410 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f426d7fa640 1 -- 172.21.15.19:0/4132718345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c035040 con 0x55b117f13410 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4294f81740 1 -- 172.21.15.19:0/4132718345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117f13410 msgr2=0x55b1182b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4294f81740 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117f13410 0x55b1182b78c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f427c02a930 tx=0x7f427c005330 comp rx=0 tx=0).stop 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4294f81740 1 -- 172.21.15.19:0/4132718345 shutdown_connections 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4294f81740 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117f13410 0x55b1182b78c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4294f81740 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117ef2730 0x55b1182b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4294f81740 1 --2- 172.21.15.19:0/4132718345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117eea550 0x55b1182aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.824+0000 7f4294f81740 1 -- 172.21.15.19:0/4132718345 >> 172.21.15.19:0/4132718345 conn(0x55b117f07560 msgr2=0x55b1182bc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 -- 172.21.15.19:0/4132718345 shutdown_connections 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 -- 172.21.15.19:0/4132718345 wait complete. 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 -- start start 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 Processor -- start 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42881ae650 con 0x55b117ef2730 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42881afac0 con 0x55b117f13410 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42881aed20 con 0x55b117eea550 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f4294f81740 10 client.0 _ll_register_callbacks cb 0x7f427c009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.825+0000 7f42877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50882/0 (socket says 172.21.15.19:50882) 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 -- 172.21.15.19:0/747825794 learned_addr learned my addr 172.21.15.19:0/747825794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4286ffd640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4286ffd640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 -- 172.21.15.19:0/747825794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 msgr2=0x55b1182ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 -- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 msgr2=0x55b1182d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182d2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 -- 172.21.15.19:0/747825794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1182d2f30 con 0x55b117ef2730 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4286ffd640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f42877fe640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182ce9a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f42880695c0 tx=0x7f4278010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc5fe71a2724f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4266ffd640 1 -- 172.21.15.19:0/747825794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780035f0 con 0x55b117ef2730 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427c0314a0 con 0x55b117ef2730 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4266ffd640 1 -- 172.21.15.19:0/747825794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f4278003790 con 0x55b117ef2730 2024-04-25T10:56:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4266ffd640 1 -- 172.21.15.19:0/747825794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f427802d050 con 0x55b117ef2730 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4294f81740 10 client.15876 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.826+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b1183378f0 con 0x55b117ef2730 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4266ffd640 1 -- 172.21.15.19:0/747825794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c0314a0 con 0x55b117ef2730 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4266ffd640 1 -- 172.21.15.19:0/747825794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f427801f6d0 con 0x55b117ef2730 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4294f81740 10 client.15876 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4294f81740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4294f81740 20 client.15876 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4294f81740 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ea7630 0x55b11833a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4294f81740 4 client.15876 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:56:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b118221580 con 0x55b117ea7630 2024-04-25T10:56:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4286ffd640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ea7630 0x55b11833a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4266ffd640 10 client.15876 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.827+0000 7f4286ffd640 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ea7630 0x55b11833a0e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634cad7b70f1c4a7 server_cookie=6957a368cbf56a01 in_seq=0 out_seq=0 2024-04-25T10:56:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4266ffd640 1 -- 172.21.15.19:0/747825794 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b118221580 con 0x55b117ea7630 2024-04-25T10:56:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4266ffd640 10 client.15876 handle_command_reply: tid=0 2024-04-25T10:56:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 client.15876 shutdown 2024-04-25T10:56:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 msgr2=0x55b1182ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182ce9a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f42880695c0 tx=0x7f4278010800 comp rx=0 tx=0).stop 2024-04-25T10:56:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 shutdown_connections 2024-04-25T10:56:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b117ea7630 0x55b11833a0e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4266ffd640 0 client.15876 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117f13410 0x55b1182d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b117ef2730 0x55b1182ce9a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 --2- 172.21.15.19:0/747825794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b117eea550 0x55b1182ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 >> 172.21.15.19:0/747825794 conn(0x55b117f07560 msgr2=0x55b1182aec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 shutdown_connections 2024-04-25T10:56:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 1 -- 172.21.15.19:0/747825794 wait complete. 2024-04-25T10:56:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.829+0000 7f4294f81740 20 client.15876 trim_cache size 0 max 16384 2024-04-25T10:56:42.841 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:56:42.841 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:56:42.841 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "e51e2b96-75f2-4c16-b322-782d6b8a5ba5", 2024-04-25T10:56:42.841 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:56:42.841 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:56:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42700afc60 msgr2=0x7f42700b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42700afc60 0x7f42700b2170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42800dd370 tx=0x7f42800ca040 comp rx=0 tx=0).stop 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 msgr2=0x7f428819c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f428819c050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f428000eb10 tx=0x7f428000c2d0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 shutdown_connections 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42700afc60 0x7f42700b2170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288108eb0 0x7f428819c050 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288108a10 0x7f428819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 --2- 172.21.15.19:0/360097794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42881031c0 0x7f428819b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 >> 172.21.15.19:0/360097794 conn(0x7f4288078110 msgr2=0x7f42881040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 shutdown_connections 2024-04-25T10:56:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.830+0000 7f428dff3640 1 -- 172.21.15.19:0/360097794 wait complete. 2024-04-25T10:56:42.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:56:42.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:56:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 -- 172.21.15.19:0/563521407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db010d380 msgr2=0x7f3db010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 --2- 172.21.15.19:0/563521407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db010d380 0x7f3db010f7a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009ca0 tx=0x7f3da802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 -- 172.21.15.19:0/563521407 shutdown_connections 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 --2- 172.21.15.19:0/563521407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db010d380 0x7f3db010f7a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 --2- 172.21.15.19:0/563521407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db0102af0 0x7f3db0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 --2- 172.21.15.19:0/563521407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3db0108a20 0x7f3db01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 -- 172.21.15.19:0/563521407 >> 172.21.15.19:0/563521407 conn(0x7f3db0078100 msgr2=0x7f3db0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 -- 172.21.15.19:0/563521407 shutdown_connections 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.958+0000 7f3db7dce640 1 -- 172.21.15.19:0/563521407 wait complete. 2024-04-25T10:56:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 Processor -- start 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 -- start start 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3db0102af0 0x7f3db019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db0108a20 0x7f3db019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db010d380 0x7f3db019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0112c00 con 0x7f3db0108a20 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0112a80 con 0x7f3db010d380 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db7dce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0112d80 con 0x7f3db0102af0 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db5369640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db0108a20 0x7f3db019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db5369640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db0108a20 0x7f3db019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50906/0 (socket says 172.21.15.19:50906) 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db5369640 1 -- 172.21.15.19:0/2500689237 learned_addr learned my addr 172.21.15.19:0/2500689237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3db0102af0 0x7f3db019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db636b640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db010d380 0x7f3db019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db5369640 1 -- 172.21.15.19:0/2500689237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3db0102af0 msgr2=0x7f3db019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db5369640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3db0102af0 0x7f3db019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.959+0000 7f3db5369640 1 -- 172.21.15.19:0/2500689237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db010d380 msgr2=0x7f3db019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3db5369640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db010d380 0x7f3db019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3db5369640 1 -- 172.21.15.19:0/2500689237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db019f510 con 0x7f3db0108a20 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3db5369640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db0108a20 0x7f3db019ac10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3da000d740 tx=0x7f3da000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3da6ffd640 1 -- 172.21.15.19:0/2500689237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0017070 con 0x7f3db0108a20 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3da6ffd640 1 -- 172.21.15.19:0/2500689237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da000de50 con 0x7f3db0108a20 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db019f800 con 0x7f3db0108a20 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3da6ffd640 1 -- 172.21.15.19:0/2500689237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0003c20 con 0x7f3db0108a20 2024-04-25T10:56:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.960+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db01a7180 con 0x7f3db0108a20 2024-04-25T10:56:42.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 Processor -- start 2024-04-25T10:56:42.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 -- start start 2024-04-25T10:56:42.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3da6ffd640 1 -- 172.21.15.19:0/2500689237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3da0003dc0 con 0x7f3db0108a20 2024-04-25T10:56:42.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00cf48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00cf4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d05d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e00c5ad80 con 0x564e0092b4d0 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e00c148b0 con 0x564e009336b0 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564e00c56d40 con 0x564e00954390 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3db5b6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00cf4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3db5b6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00cf4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50922/0 (socket says 172.21.15.19:50922) 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3db5b6a640 1 -- 172.21.15.19:0/432129026 learned_addr learned my addr 172.21.15.19:0/432129026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.963+0000 7f3db636b640 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00cf48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db5369640 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d05d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db5b6a640 1 -- 172.21.15.19:0/432129026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 msgr2=0x564e00cf48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00cf48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db5b6a640 1 -- 172.21.15.19:0/432129026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 msgr2=0x564e00d05d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d05d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db5b6a640 1 -- 172.21.15.19:0/432129026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e00be5a40 con 0x564e0092b4d0 2024-04-25T10:56:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3da6ffd640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d9c0afc60 0x7f3d9c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00cf4e30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009ca0 tx=0x7f3dac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3fb5134413b0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3db636b640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d9c0afc60 0x7f3d9c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3d9bfff640 1 -- 172.21.15.19:0/432129026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac01e070 con 0x564e0092b4d0 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3da6ffd640 1 -- 172.21.15.19:0/2500689237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3da00de990 con 0x7f3db0108a20 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3d9bfff640 1 -- 172.21.15.19:0/432129026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dac019ae0 con 0x564e0092b4d0 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.964+0000 7f3d9bfff640 1 -- 172.21.15.19:0/432129026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00e950 con 0x564e0092b4d0 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 -- 172.21.15.19:0/432129026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 msgr2=0x564e00cf4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3db636b640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d9c0afc60 0x7f3d9c0b2170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3da80094e0 tx=0x7f3da802f310 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00cf4e30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009ca0 tx=0x7f3dac011810 comp rx=0 tx=0).stop 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 -- 172.21.15.19:0/432129026 shutdown_connections 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d05d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00cf4e30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/432129026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00cf48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 -- 172.21.15.19:0/432129026 >> 172.21.15.19:0/432129026 conn(0x564e009484e0 msgr2=0x564e00cf1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 -- 172.21.15.19:0/432129026 shutdown_connections 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.965+0000 7f3dbec7f740 1 -- 172.21.15.19:0/432129026 wait complete. 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 -- start start 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 Processor -- start 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00d03a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d03f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d09900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e00d063f0 con 0x564e0092b4d0 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e00be1d10 con 0x564e009336b0 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564e00d06570 con 0x564e00954390 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3dbec7f740 10 client.0 _ll_register_callbacks cb 0x564e00c0e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3db5b6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00d03a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3db5b6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00d03a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50934/0 (socket says 172.21.15.19:50934) 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3db5b6a640 1 -- 172.21.15.19:0/3779373989 learned_addr learned my addr 172.21.15.19:0/3779373989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.966+0000 7f3db5369640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d03f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db5b6a640 1 -- 172.21.15.19:0/3779373989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 msgr2=0x564e00d09900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db636b640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d09900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d09900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db5b6a640 1 -- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 msgr2=0x564e00d03f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d03f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db5b6a640 1 -- 172.21.15.19:0/3779373989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e00a367d0 con 0x564e0092b4d0 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db636b640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d09900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00d03a00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009f50 tx=0x7f3dac00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69ff27b435464c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3d997fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3d997fa640 1 -- 172.21.15.19:0/3779373989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac027020 con 0x564e0092b4d0 2024-04-25T10:56:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3d997fa640 1 -- 172.21.15.19:0/3779373989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3dac005460 con 0x564e0092b4d0 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dac019ae0 con 0x564e0092b4d0 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3d997fa640 1 -- 172.21.15.19:0/3779373989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3dac032050 con 0x564e0092b4d0 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3dbec7f740 10 client.15888 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564e0096c3f0 con 0x564e0092b4d0 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.967+0000 7f3d997fa640 1 -- 172.21.15.19:0/3779373989 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dac019ae0 con 0x564e0092b4d0 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3d997fa640 1 -- 172.21.15.19:0/3779373989 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3dac00ea00 con 0x564e0092b4d0 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3dbec7f740 10 client.15888 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3dbec7f740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3dbec7f740 20 client.15888 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e00912f60 0x564e00d15ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3dbec7f740 4 client.15888 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e00c68990 con 0x564e00912f60 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3db5369640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e00912f60 0x564e00d15ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3d997fa640 10 client.15888 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.968+0000 7f3db5369640 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e00912f60 0x564e00d15ad0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ccb2b4ae9e39c85 server_cookie=ae2b4dca8155dbe1 in_seq=0 out_seq=0 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3d997fa640 1 -- 172.21.15.19:0/3779373989 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e00c68990 con 0x564e00912f60 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3d997fa640 10 client.15888 handle_command_reply: tid=0 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 client.15888 shutdown 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 msgr2=0x564e00d03a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00d03a00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009f50 tx=0x7f3dac00e3b0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 shutdown_connections 2024-04-25T10:56:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e00912f60 0x564e00d15ad0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d09900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3d997fa640 0 client.15888 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e009336b0 0x564e00d03f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/3779373989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e0092b4d0 0x564e00d03a00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.969+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 >> 172.21.15.19:0/3779373989 conn(0x564e009484e0 msgr2=0x564e00cf6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.970+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 shutdown_connections 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.970+0000 7f3dbec7f740 1 -- 172.21.15.19:0/3779373989 wait complete. 2024-04-25T10:56:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.970+0000 7f3dbec7f740 20 client.15888 trim_cache size 0 max 16384 2024-04-25T10:56:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 Processor -- start 2024-04-25T10:56:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 -- start start 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d05620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e009336b0 0x564e00d05b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d1b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01ab630 con 0x564e009336b0 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01adeb0 con 0x564e0092b4d0 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01abca0 con 0x564e00954390 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e009336b0 0x564e00d05b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e009336b0 0x564e00d05b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50940/0 (socket says 172.21.15.19:50940) 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 -- 172.21.15.19:0/686750520 learned_addr learned my addr 172.21.15.19:0/686750520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db636b640 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d1b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d05620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 -- 172.21.15.19:0/686750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 msgr2=0x564e00d1b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d1b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 -- 172.21.15.19:0/686750520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 msgr2=0x564e00d05620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d05620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5369640 1 -- 172.21.15.19:0/686750520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0092c0f0 con 0x564e009336b0 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db636b640 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d1b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.988+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d05620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.989+0000 7f3db5369640 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e009336b0 0x564e00d05b60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009610 tx=0x7f3da00e0ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573d187c7b322992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.989+0000 7f3d9b7fe640 1 -- 172.21.15.19:0/686750520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0020040 con 0x564e009336b0 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.989+0000 7f3d9b7fe640 1 -- 172.21.15.19:0/686750520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da00af300 con 0x564e009336b0 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.989+0000 7f3d9b7fe640 1 -- 172.21.15.19:0/686750520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da00dc390 con 0x564e009336b0 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 -- 172.21.15.19:0/686750520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e009336b0 msgr2=0x564e00d05b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e009336b0 0x564e00d05b60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009610 tx=0x7f3da00e0ea0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 -- 172.21.15.19:0/686750520 shutdown_connections 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e00954390 0x564e00d1b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e009336b0 0x564e00d05b60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/686750520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d05620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 -- 172.21.15.19:0/686750520 >> 172.21.15.19:0/686750520 conn(0x564e009484e0 msgr2=0x564e00d04f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 -- 172.21.15.19:0/686750520 shutdown_connections 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 -- 172.21.15.19:0/686750520 wait complete. 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 -- start start 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 Processor -- start 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.990+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d0aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e009336b0 0x564e00d0b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3dbec7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e00954390 0x564e00d7bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01ac370 con 0x564e00954390 2024-04-25T10:56:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01ad7e0 con 0x564e0092b4d0 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3dbec7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01aca40 con 0x564e009336b0 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3dbec7f740 10 client.0 _ll_register_callbacks cb 0x7f3dac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e00954390 0x564e00d7bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e00954390 0x564e00d7bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50952/0 (socket says 172.21.15.19:50952) 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 -- 172.21.15.19:0/379248392 learned_addr learned my addr 172.21.15.19:0/379248392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d0aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db5369640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e009336b0 0x564e00d0b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 -- 172.21.15.19:0/379248392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e009336b0 msgr2=0x564e00d0b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e009336b0 0x564e00d0b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 -- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 msgr2=0x564e00d0aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d0aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 -- 172.21.15.19:0/379248392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e00c1b4b0 con 0x564e00954390 2024-04-25T10:56:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d0aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db5369640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e009336b0 0x564e00d0b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3db636b640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e00954390 0x564e00d7bcd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009ca0 tx=0x7f3da802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d35d0e6fb9c33a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3d997fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3d997fa640 1 -- 172.21.15.19:0/379248392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8057060 con 0x564e00954390 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3d997fa640 1 -- 172.21.15.19:0/379248392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3da8003d00 con 0x564e00954390 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.991+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e00d02ea0 con 0x564e00954390 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3d997fa640 1 -- 172.21.15.19:0/379248392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3da8004040 con 0x564e00954390 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 10 client.15894 fetch_fsmap learned FSMap version 20 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564e00d7cef0 con 0x564e00954390 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3d997fa640 1 -- 172.21.15.19:0/379248392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564e00d02ea0 con 0x564e00954390 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3d997fa640 1 -- 172.21.15.19:0/379248392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3da802fb80 con 0x564e00954390 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 10 client.15894 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:56:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 20 client.15894 populate_metadata read hostname 'smithi019' 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e008e83a0 0x564e00d7f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 4 client.15894 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e00c62800 con 0x564e008e83a0 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.992+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e008e83a0 0x564e00d7f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3d997fa640 10 client.15894 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3db5b6a640 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e008e83a0 0x564e00d7f6e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41789453a0c42cb server_cookie=4d33997eb8776fda in_seq=0 out_seq=0 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3d997fa640 1 -- 172.21.15.19:0/379248392 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564e00c62800 con 0x564e008e83a0 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3d997fa640 10 client.15894 handle_command_reply: tid=0 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 client.15894 shutdown 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e00954390 msgr2=0x564e00d7bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e00954390 0x564e00d7bcd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009ca0 tx=0x7f3da802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 shutdown_connections 2024-04-25T10:56:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564e008e83a0 0x564e00d7f6e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e00954390 0x564e00d7bcd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e009336b0 0x564e00d0b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 --2- 172.21.15.19:0/379248392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0092b4d0 0x564e00d0aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3d997fa640 0 client.15894 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:56:43.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 >> 172.21.15.19:0/379248392 conn(0x564e009484e0 msgr2=0x564e00d02720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:43.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 shutdown_connections 2024-04-25T10:56:43.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 1 -- 172.21.15.19:0/379248392 wait complete. 2024-04-25T10:56:43.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.993+0000 7f3dbec7f740 20 client.15894 trim_cache size 0 max 16384 2024-04-25T10:56:43.006 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:56:43.006 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:56:43.006 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:56:43.006 INFO:teuthology.orchestra.run.smithi019.stdout: "e51e2b96-75f2-4c16-b322-782d6b8a5ba5": { 2024-04-25T10:56:43.007 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:56:43.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "e51e2b96-75f2-4c16-b322-782d6b8a5ba5", 2024-04-25T10:56:43.007 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:56:43.007 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:56:43.007 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:56:43.007 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:56:43.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d9c0afc60 msgr2=0x7f3d9c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:43.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d9c0afc60 0x7f3d9c0b2170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3da80094e0 tx=0x7f3da802f310 comp rx=0 tx=0).stop 2024-04-25T10:56:43.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db0108a20 msgr2=0x7f3db019ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db0108a20 0x7f3db019ac10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3da000d740 tx=0x7f3da000dc00 comp rx=0 tx=0).stop 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 shutdown_connections 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d9c0afc60 0x7f3d9c0b2170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db010d380 0x7f3db019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db0108a20 0x7f3db019ac10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 --2- 172.21.15.19:0/2500689237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3db0102af0 0x7f3db019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 >> 172.21.15.19:0/2500689237 conn(0x7f3db0078100 msgr2=0x7f3db0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 shutdown_connections 2024-04-25T10:56:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:56:42.994+0000 7f3db7dce640 1 -- 172.21.15.19:0/2500689237 wait complete. 2024-04-25T10:56:43.009 INFO:tasks.cephfs.filesystem:scrub status for tag:e51e2b96-75f2-4c16-b322-782d6b8a5ba5 - {'path': '/', 'tag': 'e51e2b96-75f2-4c16-b322-782d6b8a5ba5', 'options': 'recursive,force'} 2024-04-25T10:56:43.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:43 smithi112 ceph-mon[32799]: pgmap v69: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 211 KiB/s wr, 77 op/s 2024-04-25T10:56:43.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:43 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:56:43.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:43 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:43.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:43 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:56:43.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:43 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:43.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:43 smithi019 ceph-mon[29209]: pgmap v69: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 211 KiB/s wr, 77 op/s 2024-04-25T10:56:43.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:43 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:56:43.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:43 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:43.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:43 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:56:43.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:43 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:43 smithi169 ceph-mon[33049]: pgmap v69: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 211 KiB/s wr, 77 op/s 2024-04-25T10:56:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:43 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:56:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:43 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:43 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:56:43.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:43 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:56:45.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:45 smithi112 ceph-mon[32799]: pgmap v70: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 3 op/s 2024-04-25T10:56:45.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:45 smithi019 ceph-mon[29209]: pgmap v70: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 3 op/s 2024-04-25T10:56:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:45 smithi169 ceph-mon[33049]: pgmap v70: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 3 op/s 2024-04-25T10:56:47.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:47 smithi112 ceph-mon[32799]: pgmap v71: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-04-25T10:56:47.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:47 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:56:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:47 smithi019 ceph-mon[29209]: pgmap v71: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-04-25T10:56:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:47 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:56:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:47 smithi169 ceph-mon[33049]: pgmap v71: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-04-25T10:56:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:47 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:56:49.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:49 smithi112 ceph-mon[32799]: pgmap v72: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-04-25T10:56:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:49 smithi019 ceph-mon[29209]: pgmap v72: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-04-25T10:56:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:49 smithi169 ceph-mon[33049]: pgmap v72: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-04-25T10:56:51.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:51 smithi112 ceph-mon[32799]: pgmap v73: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-04-25T10:56:51.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:51 smithi019 ceph-mon[29209]: pgmap v73: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-04-25T10:56:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:51 smithi169 ceph-mon[33049]: pgmap v73: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-04-25T10:56:53.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:53 smithi112 ceph-mon[32799]: pgmap v74: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-04-25T10:56:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:53 smithi019 ceph-mon[29209]: pgmap v74: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-04-25T10:56:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:53 smithi169 ceph-mon[33049]: pgmap v74: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-04-25T10:56:55.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:55 smithi112 ceph-mon[32799]: pgmap v75: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T10:56:55.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:55 smithi019 ceph-mon[29209]: pgmap v75: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T10:56:55.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:55 smithi169 ceph-mon[33049]: pgmap v75: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T10:56:57.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:57 smithi112 ceph-mon[32799]: pgmap v76: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T10:56:57.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:57 smithi019 ceph-mon[29209]: pgmap v76: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T10:56:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:57 smithi169 ceph-mon[33049]: pgmap v76: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T10:56:59.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:56:59 smithi112 ceph-mon[32799]: pgmap v77: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-04-25T10:56:59.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:56:59 smithi019 ceph-mon[29209]: pgmap v77: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-04-25T10:56:59.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:56:59 smithi169 ceph-mon[33049]: pgmap v77: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-04-25T10:57:01.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:01 smithi112 ceph-mon[32799]: pgmap v78: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-04-25T10:57:01.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:01 smithi019 ceph-mon[29209]: pgmap v78: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-04-25T10:57:01.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:01 smithi169 ceph-mon[33049]: pgmap v78: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 1 op/s 2024-04-25T10:57:03.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:03 smithi112 ceph-mon[32799]: pgmap v79: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:03.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:03 smithi019 ceph-mon[29209]: pgmap v79: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:03 smithi169 ceph-mon[33049]: pgmap v79: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:05.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:05 smithi019 ceph-mon[29209]: pgmap v80: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:05.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:05 smithi169 ceph-mon[33049]: pgmap v80: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:05.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:05 smithi112 ceph-mon[32799]: pgmap v80: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:07.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:07 smithi019 ceph-mon[29209]: pgmap v81: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:07 smithi169 ceph-mon[33049]: pgmap v81: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:07.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:07 smithi112 ceph-mon[32799]: pgmap v81: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:09.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:09 smithi019 ceph-mon[29209]: pgmap v82: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:09.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:09 smithi169 ceph-mon[33049]: pgmap v82: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:09.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:09 smithi112 ceph-mon[32799]: pgmap v82: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:11.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:11 smithi019 ceph-mon[29209]: pgmap v83: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:11.361 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:11 smithi112 ceph-mon[32799]: pgmap v83: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:11.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:11 smithi169 ceph-mon[33049]: pgmap v83: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:57:13.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:57:13.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:57:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 -- 172.21.15.19:0/2272365686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1190104bf0 msgr2=0x7f1190105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 --2- 172.21.15.19:0/2272365686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1190104bf0 0x7f1190105000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f118c009900 tx=0x7f118c02a410 comp rx=0 tx=0).stop 2024-04-25T10:57:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 -- 172.21.15.19:0/2272365686 shutdown_connections 2024-04-25T10:57:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 --2- 172.21.15.19:0/2272365686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f119010e040 0x7f1190110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 --2- 172.21.15.19:0/2272365686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f119010b650 0x7f119010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 --2- 172.21.15.19:0/2272365686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1190104bf0 0x7f1190105000 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 -- 172.21.15.19:0/2272365686 >> 172.21.15.19:0/2272365686 conn(0x7f1190078100 msgr2=0x7f1190078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 -- 172.21.15.19:0/2272365686 shutdown_connections 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.139+0000 7f1198092640 1 -- 172.21.15.19:0/2272365686 wait complete. 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 Processor -- start 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 -- start start 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1190104bf0 0x7f119019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f119010b650 0x7f119019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f119010e040 0x7f119019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1190113cf0 con 0x7f119010b650 2024-04-25T10:57:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1190113b70 con 0x7f1190104bf0 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.140+0000 7f1198092640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1190113e70 con 0x7f119010e040 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f119010b650 0x7f119019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f119010b650 0x7f119019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43322/0 (socket says 172.21.15.19:43322) 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 -- 172.21.15.19:0/1272934686 learned_addr learned my addr 172.21.15.19:0/1272934686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f1195e2e640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1190104bf0 0x7f119019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119662f640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f119010e040 0x7f119019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 -- 172.21.15.19:0/1272934686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f119010e040 msgr2=0x7f119019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f119010e040 0x7f119019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 -- 172.21.15.19:0/1272934686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1190104bf0 msgr2=0x7f119019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1190104bf0 0x7f119019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 -- 172.21.15.19:0/1272934686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119019f5b0 con 0x7f119010b650 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119662f640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f119010e040 0x7f119019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f1195e2e640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1190104bf0 0x7f119019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f119562d640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f119010b650 0x7f119019acb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f118000d770 tx=0x7f118000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f1186ffd640 1 -- 172.21.15.19:0/1272934686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180017070 con 0x7f119010b650 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f1186ffd640 1 -- 172.21.15.19:0/1272934686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11800054d0 con 0x7f119010b650 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f1186ffd640 1 -- 172.21.15.19:0/1272934686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180004c30 con 0x7f119010b650 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119019f8a0 con 0x7f119010b650 2024-04-25T10:57:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.141+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11901a7220 con 0x7f119010b650 2024-04-25T10:57:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 Processor -- start 2024-04-25T10:57:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 -- start start 2024-04-25T10:57:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f1186ffd640 1 -- 172.21.15.19:0/1272934686 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1180004e10 con 0x7f119010b650 2024-04-25T10:57:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae20930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae20e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596bad86d80 con 0x5596baa574d0 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5596bad408b0 con 0x5596baa80390 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.144+0000 7f119f01d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5596bad82d40 con 0x5596baa5f6b0 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae20e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae20e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43338/0 (socket says 172.21.15.19:43338) 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 -- 172.21.15.19:0/4206914473 learned_addr learned my addr 172.21.15.19:0/4206914473 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f119562d640 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f119662f640 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae20930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1186ffd640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f117c0afce0 0x7f117c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 -- 172.21.15.19:0/4206914473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 msgr2=0x5596bae31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 -- 172.21.15.19:0/4206914473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 msgr2=0x5596bae20930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae20930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 -- 172.21.15.19:0/4206914473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596bad11a40 con 0x5596baa574d0 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f119662f640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f117c0afce0 0x7f117c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1186ffd640 1 -- 172.21.15.19:0/1272934686 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f11800deb30 con 0x7f119010b650 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f1195e2e640 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae20e70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f118c009900 tx=0x7f118c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1743e183f60615ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.145+0000 7f117bfff640 1 -- 172.21.15.19:0/4206914473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c035030 con 0x5596baa574d0 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f117bfff640 1 -- 172.21.15.19:0/4206914473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118c005de0 con 0x5596baa574d0 2024-04-25T10:57:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f117bfff640 1 -- 172.21.15.19:0/4206914473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0035f0 con 0x5596baa574d0 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119f01d740 1 -- 172.21.15.19:0/4206914473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 msgr2=0x5596bae20e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119f01d740 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae20e70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f118c009900 tx=0x7f118c004390 comp rx=0 tx=0).stop 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119662f640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f117c0afce0 0x7f117c0b21f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1188009370 tx=0x7f1188004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119f01d740 1 -- 172.21.15.19:0/4206914473 shutdown_connections 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119f01d740 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119f01d740 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae20e70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119f01d740 1 --2- 172.21.15.19:0/4206914473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae20930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.146+0000 7f119f01d740 1 -- 172.21.15.19:0/4206914473 >> 172.21.15.19:0/4206914473 conn(0x5596baa744e0 msgr2=0x5596bae1d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.147+0000 7f119f01d740 1 -- 172.21.15.19:0/4206914473 shutdown_connections 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.147+0000 7f119f01d740 1 -- 172.21.15.19:0/4206914473 wait complete. 2024-04-25T10:57:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.147+0000 7f119f01d740 1 -- start start 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.147+0000 7f119f01d740 1 Processor -- start 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa574d0 0x5596bae2f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa5f6b0 0x5596bae2feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae3b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596bae32420 con 0x5596baa5f6b0 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5596bad0dd10 con 0x5596baa574d0 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5596bae325a0 con 0x5596baa80390 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 10 client.0 _ll_register_callbacks cb 0x5596bad3a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa5f6b0 0x5596bae2feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f1195e2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa574d0 0x5596bae2f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa5f6b0 0x5596bae2feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43342/0 (socket says 172.21.15.19:43342) 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 -- 172.21.15.19:0/1660586483 learned_addr learned my addr 172.21.15.19:0/1660586483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119662f640 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae3b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 -- 172.21.15.19:0/1660586483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 msgr2=0x5596bae3b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae3b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 -- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa574d0 msgr2=0x5596bae2f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa574d0 0x5596bae2f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 -- 172.21.15.19:0/1660586483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596bab627d0 con 0x5596baa5f6b0 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f1195e2e640 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa574d0 0x5596bae2f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119562d640 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa5f6b0 0x5596bae2feb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f11800dc310 tx=0x7f11800e2270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15199b8333fb95cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f11797fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f11797fa640 1 -- 172.21.15.19:0/1660586483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180020070 con 0x5596baa5f6b0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f11797fa640 1 -- 172.21.15.19:0/1660586483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f11800af6d0 con 0x5596baa5f6b0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.148+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596bad94990 con 0x5596baa5f6b0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f11797fa640 1 -- 172.21.15.19:0/1660586483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1180005680 con 0x5596baa5f6b0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 10 client.15906 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5596baa983f0 con 0x5596baa5f6b0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f11797fa640 1 -- 172.21.15.19:0/1660586483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5596bad94990 con 0x5596baa5f6b0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f11797fa640 1 -- 172.21.15.19:0/1660586483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f11800c1210 con 0x5596baa5f6b0 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 10 client.15906 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 20 client.15906 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa3ef60 0x5596bae41a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 4 client.15906 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596bad8e800 con 0x5596baa3ef60 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.149+0000 7f1195e2e640 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa3ef60 0x5596bae41a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.150+0000 7f11797fa640 10 client.15906 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.150+0000 7f1195e2e640 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa3ef60 0x5596bae41a90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69926be4718b557 server_cookie=1106f12b796d597 in_seq=0 out_seq=0 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f11797fa640 1 -- 172.21.15.19:0/1660586483 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596bad8e800 con 0x5596baa3ef60 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f11797fa640 10 client.15906 handle_command_reply: tid=0 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 client.15906 shutdown 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa5f6b0 msgr2=0x5596bae2feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa5f6b0 0x5596bae2feb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f11800dc310 tx=0x7f11800e2270 comp rx=0 tx=0).stop 2024-04-25T10:57:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 shutdown_connections 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa3ef60 0x5596bae41a90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae3b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa5f6b0 0x5596bae2feb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f11797fa640 0 client.15906 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 --2- 172.21.15.19:0/1660586483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa574d0 0x5596bae2f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 >> 172.21.15.19:0/1660586483 conn(0x5596baa744e0 msgr2=0x5596bae22710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 shutdown_connections 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 1 -- 172.21.15.19:0/1660586483 wait complete. 2024-04-25T10:57:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.151+0000 7f119f01d740 20 client.15906 trim_cache size 0 max 16384 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.169+0000 7f119f01d740 1 Processor -- start 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.169+0000 7f119f01d740 1 -- start start 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119f01d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119f01d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119f01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119f01d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11901ab6a0 con 0x5596baa574d0 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119f01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11901adf00 con 0x5596baa80390 2024-04-25T10:57:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119f01d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11901abcf0 con 0x5596baa5f6b0 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43346/0 (socket says 172.21.15.19:43346) 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 -- 172.21.15.19:0/2594747647 learned_addr learned my addr 172.21.15.19:0/2594747647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119562d640 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 -- 172.21.15.19:0/2594747647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 msgr2=0x5596bae37d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae37d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119662f640 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 -- 172.21.15.19:0/2594747647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 msgr2=0x5596bae226d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae226d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 -- 172.21.15.19:0/2594747647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596baa580f0 con 0x5596baa574d0 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f119662f640 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f1195e2e640 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae377e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f118c006010 tx=0x7f118c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866077bfb52bf13c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f117b7fe640 1 -- 172.21.15.19:0/2594747647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c035040 con 0x5596baa574d0 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f117b7fe640 1 -- 172.21.15.19:0/2594747647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118c004f90 con 0x5596baa574d0 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.170+0000 7f117b7fe640 1 -- 172.21.15.19:0/2594747647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c004cf0 con 0x5596baa574d0 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 -- 172.21.15.19:0/2594747647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 msgr2=0x5596bae377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae377e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f118c006010 tx=0x7f118c004c80 comp rx=0 tx=0).stop 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 -- 172.21.15.19:0/2594747647 shutdown_connections 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa80390 0x5596bae226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa5f6b0 0x5596bae37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 --2- 172.21.15.19:0/2594747647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae377e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 -- 172.21.15.19:0/2594747647 >> 172.21.15.19:0/2594747647 conn(0x5596baa744e0 msgr2=0x5596bae2e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 -- 172.21.15.19:0/2594747647 shutdown_connections 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.171+0000 7f119f01d740 1 -- 172.21.15.19:0/2594747647 wait complete. 2024-04-25T10:57:13.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 -- start start 2024-04-25T10:57:13.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 Processor -- start 2024-04-25T10:57:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa5f6b0 0x5596bae32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae33230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11901ac3c0 con 0x5596baa574d0 2024-04-25T10:57:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11901ad830 con 0x5596baa5f6b0 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11901aca90 con 0x5596baa80390 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119f01d740 10 client.0 _ll_register_callbacks cb 0x7f118000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f1195e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f1195e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43352/0 (socket says 172.21.15.19:43352) 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f1195e2e640 1 -- 172.21.15.19:0/2468834514 learned_addr learned my addr 172.21.15.19:0/2468834514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.172+0000 7f119562d640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa5f6b0 0x5596bae32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f1195e2e640 1 -- 172.21.15.19:0/2468834514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 msgr2=0x5596bae33230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f119662f640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae33230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f1195e2e640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae33230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f1195e2e640 1 -- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa5f6b0 msgr2=0x5596bae32cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f1195e2e640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa5f6b0 0x5596bae32cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f1195e2e640 1 -- 172.21.15.19:0/2468834514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596bad474b0 con 0x5596baa574d0 2024-04-25T10:57:13.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f119562d640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa5f6b0 0x5596bae32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f1195e2e640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae327b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f118c02a930 tx=0x7f118c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7694f201fee6bc75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f11797fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f11797fa640 1 -- 172.21.15.19:0/2468834514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c035040 con 0x5596baa574d0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f11797fa640 1 -- 172.21.15.19:0/2468834514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f118c050630 con 0x5596baa574d0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118c004f90 con 0x5596baa574d0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f11797fa640 1 -- 172.21.15.19:0/2468834514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f118c050830 con 0x5596baa574d0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f119f01d740 10 client.15912 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5596baea8f60 con 0x5596baa574d0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f11797fa640 1 -- 172.21.15.19:0/2468834514 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118c004f90 con 0x5596baa574d0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f11797fa640 1 -- 172.21.15.19:0/2468834514 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f118c05a4a0 con 0x5596baa574d0 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.173+0000 7f119f01d740 10 client.15912 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:13.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119f01d740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119f01d740 20 client.15912 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119f01d740 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa143a0 0x5596baeab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119f01d740 4 client.15912 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596bad094a0 con 0x5596baa143a0 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119562d640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa143a0 0x5596baeab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119562d640 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa143a0 0x5596baeab1f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27efe9ad5025236 server_cookie=38f1c5562ddf0c37 in_seq=0 out_seq=0 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f11797fa640 10 client.15912 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f11797fa640 1 -- 172.21.15.19:0/2468834514 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5596bad094a0 con 0x5596baa143a0 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f11797fa640 10 client.15912 handle_command_reply: tid=0 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.174+0000 7f119f01d740 1 client.15912 shutdown 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 msgr2=0x5596bae327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae327b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f118c02a930 tx=0x7f118c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 shutdown_connections 2024-04-25T10:57:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5596baa143a0 0x5596baeab1f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5596baa80390 0x5596bae33230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596baa5f6b0 0x5596bae32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f11797fa640 0 client.15912 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 --2- 172.21.15.19:0/2468834514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596baa574d0 0x5596bae327b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 >> 172.21.15.19:0/2468834514 conn(0x5596baa744e0 msgr2=0x5596bae226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 shutdown_connections 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 1 -- 172.21.15.19:0/2468834514 wait complete. 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f119f01d740 20 client.15912 trim_cache size 0 max 16384 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:57:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:57:13.185 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f117c0afce0 msgr2=0x7f117c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f117c0afce0 0x7f117c0b21f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1188009370 tx=0x7f1188004a40 comp rx=0 tx=0).stop 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f119010b650 msgr2=0x7f119019acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f119010b650 0x7f119019acb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f118000d770 tx=0x7f118000dc30 comp rx=0 tx=0).stop 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 shutdown_connections 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f117c0afce0 0x7f117c0b21f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f119010e040 0x7f119019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f119010b650 0x7f119019acb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 --2- 172.21.15.19:0/1272934686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1190104bf0 0x7f119019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.175+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 >> 172.21.15.19:0/1272934686 conn(0x7f1190078100 msgr2=0x7f1190101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.176+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 shutdown_connections 2024-04-25T10:57:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.176+0000 7f1198092640 1 -- 172.21.15.19:0/1272934686 wait complete. 2024-04-25T10:57:13.188 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:57:13.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:57:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 -- 172.21.15.19:0/4164188138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248106990 msgr2=0x7f9248106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 --2- 172.21.15.19:0/4164188138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248106990 0x7f9248106dc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9238009900 tx=0x7f923802a410 comp rx=0 tx=0).stop 2024-04-25T10:57:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 -- 172.21.15.19:0/4164188138 shutdown_connections 2024-04-25T10:57:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 --2- 172.21.15.19:0/4164188138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248107450 0x7f924810ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 --2- 172.21.15.19:0/4164188138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248106990 0x7f9248106dc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 --2- 172.21.15.19:0/4164188138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9248102400 0x7f9248102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 -- 172.21.15.19:0/4164188138 >> 172.21.15.19:0/4164188138 conn(0x7f92480fd860 msgr2=0x7f92480ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 -- 172.21.15.19:0/4164188138 shutdown_connections 2024-04-25T10:57:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.300+0000 7f924ea1a640 1 -- 172.21.15.19:0/4164188138 wait complete. 2024-04-25T10:57:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 Processor -- start 2024-04-25T10:57:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 -- start start 2024-04-25T10:57:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9248102400 0x7f924819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248106990 0x7f924819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248107450 0x7f924819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9248112be0 con 0x7f9248107450 2024-04-25T10:57:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9248112a60 con 0x7f9248102400 2024-04-25T10:57:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924ea1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9248112d60 con 0x7f9248106990 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924cfb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248107450 0x7f924819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924cfb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248107450 0x7f924819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43364/0 (socket says 172.21.15.19:43364) 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f924cfb7640 1 -- 172.21.15.19:0/773406912 learned_addr learned my addr 172.21.15.19:0/773406912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f9247fff640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9248102400 0x7f924819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.301+0000 7f92477fe640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248106990 0x7f924819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924cfb7640 1 -- 172.21.15.19:0/773406912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248106990 msgr2=0x7f924819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924cfb7640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248106990 0x7f924819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924cfb7640 1 -- 172.21.15.19:0/773406912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9248102400 msgr2=0x7f924819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924cfb7640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9248102400 0x7f924819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924cfb7640 1 -- 172.21.15.19:0/773406912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924819f590 con 0x7f9248107450 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924cfb7640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248107450 0x7f924819ef50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f924000eaa0 tx=0x7f924000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f92457fa640 1 -- 172.21.15.19:0/773406912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924000cdb0 con 0x7f9248107450 2024-04-25T10:57:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f92457fa640 1 -- 172.21.15.19:0/773406912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9240005020 con 0x7f9248107450 2024-04-25T10:57:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924819f880 con 0x7f9248107450 2024-04-25T10:57:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f92457fa640 1 -- 172.21.15.19:0/773406912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92400116d0 con 0x7f9248107450 2024-04-25T10:57:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.302+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a7200 con 0x7f9248107450 2024-04-25T10:57:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.303+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924806a190 con 0x7f9248107450 2024-04-25T10:57:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.306+0000 7f92457fa640 1 -- 172.21.15.19:0/773406912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f92400051d0 con 0x7f9248107450 2024-04-25T10:57:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.306+0000 7f92457fa640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92340afc60 0x7f92340b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.306+0000 7f9247fff640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92340afc60 0x7f92340b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.306+0000 7f92457fa640 1 -- 172.21.15.19:0/773406912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f92400df630 con 0x7f9248107450 2024-04-25T10:57:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.307+0000 7f9247fff640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92340afc60 0x7f92340b2170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9230005fd0 tx=0x7f9230005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.310+0000 7f92457fa640 1 -- 172.21.15.19:0/773406912 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9240016030 con 0x7f9248107450 2024-04-25T10:57:13.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:13 smithi019 ceph-mon[29209]: pgmap v84: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:13.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:13 smithi169 ceph-mon[33049]: pgmap v84: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:13.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.538+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f924819ba80 con 0x7f9248107450 2024-04-25T10:57:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.538+0000 7f92457fa640 1 -- 172.21.15.19:0/773406912 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f92400a7ea0 con 0x7f9248107450 2024-04-25T10:57:13.540 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:57:13.540 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:57:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:57:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92340afc60 msgr2=0x7f92340b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92340afc60 0x7f92340b2170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9230005fd0 tx=0x7f9230005850 comp rx=0 tx=0).stop 2024-04-25T10:57:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248107450 msgr2=0x7f924819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248107450 0x7f924819ef50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f924000eaa0 tx=0x7f924000ef60 comp rx=0 tx=0).stop 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 shutdown_connections 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92340afc60 0x7f92340b2170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9248107450 0x7f924819ef50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248106990 0x7f924819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 --2- 172.21.15.19:0/773406912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9248102400 0x7f924819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.540+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 >> 172.21.15.19:0/773406912 conn(0x7f92480fd860 msgr2=0x7f924810f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.541+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 shutdown_connections 2024-04-25T10:57:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.541+0000 7f924ea1a640 1 -- 172.21.15.19:0/773406912 wait complete. 2024-04-25T10:57:13.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:57:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:13 smithi112 ceph-mon[32799]: pgmap v84: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:57:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 -- 172.21.15.19:0/3278960446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108ec0 msgr2=0x7f1980102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 --2- 172.21.15.19:0/3278960446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108ec0 0x7f1980102b60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1974009900 tx=0x7f197402a410 comp rx=0 tx=0).stop 2024-04-25T10:57:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 -- 172.21.15.19:0/3278960446 shutdown_connections 2024-04-25T10:57:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 --2- 172.21.15.19:0/3278960446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19801031d0 0x7f198010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 --2- 172.21.15.19:0/3278960446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108ec0 0x7f1980102b60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 --2- 172.21.15.19:0/3278960446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1980108a20 0x7f1980102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 -- 172.21.15.19:0/3278960446 >> 172.21.15.19:0/3278960446 conn(0x7f1980078180 msgr2=0x7f19800785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 -- 172.21.15.19:0/3278960446 shutdown_connections 2024-04-25T10:57:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.666+0000 7f198641e640 1 -- 172.21.15.19:0/3278960446 wait complete. 2024-04-25T10:57:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 Processor -- start 2024-04-25T10:57:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 -- start start 2024-04-25T10:57:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19801031d0 0x7f198019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108a20 0x7f198019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1980108ec0 0x7f198019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1980112c20 con 0x7f1980108a20 2024-04-25T10:57:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1980112aa0 con 0x7f19801031d0 2024-04-25T10:57:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f198641e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1980112da0 con 0x7f1980108ec0 2024-04-25T10:57:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f197f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108a20 0x7f198019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.667+0000 7f197f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108a20 0x7f198019acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43378/0 (socket says 172.21.15.19:43378) 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197f7fe640 1 -- 172.21.15.19:0/2571259451 learned_addr learned my addr 172.21.15.19:0/2571259451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197ffff640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19801031d0 0x7f198019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f19849bb640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1980108ec0 0x7f198019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197f7fe640 1 -- 172.21.15.19:0/2571259451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1980108ec0 msgr2=0x7f198019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197f7fe640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1980108ec0 0x7f198019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197f7fe640 1 -- 172.21.15.19:0/2571259451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19801031d0 msgr2=0x7f198019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197f7fe640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19801031d0 0x7f198019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197f7fe640 1 -- 172.21.15.19:0/2571259451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198019f570 con 0x7f1980108a20 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197f7fe640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108a20 0x7f198019acf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1974032fd0 tx=0x7f1974004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197d7fa640 1 -- 172.21.15.19:0/2571259451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197403f030 con 0x7f1980108a20 2024-04-25T10:57:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197d7fa640 1 -- 172.21.15.19:0/2571259451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1974004490 con 0x7f1980108a20 2024-04-25T10:57:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198019f800 con 0x7f1980108a20 2024-04-25T10:57:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f197d7fa640 1 -- 172.21.15.19:0/2571259451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974035070 con 0x7f1980108a20 2024-04-25T10:57:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.668+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198019fd40 con 0x7f1980108a20 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 Processor -- start 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f197d7fa640 1 -- 172.21.15.19:0/2571259451 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1974042030 con 0x7f1980108a20 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 -- start start 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13f2390 0x55beb1793810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb1793d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13d16b0 0x55beb17a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55beb16f8d80 con 0x55beb13d16b0 2024-04-25T10:57:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55beb16b2a20 con 0x55beb13f2390 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f198d3a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55beb16f4d40 con 0x55beb13c94d0 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f197f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13d16b0 0x55beb17a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f197f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13d16b0 0x55beb17a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43392/0 (socket says 172.21.15.19:43392) 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f197f7fe640 1 -- 172.21.15.19:0/142621371 learned_addr learned my addr 172.21.15.19:0/142621371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f197ffff640 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb1793d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.671+0000 7f19849bb640 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13f2390 0x55beb1793810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197d7fa640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19680afce0 0x7f19680b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197f7fe640 1 -- 172.21.15.19:0/142621371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 msgr2=0x55beb1793d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197f7fe640 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb1793d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197f7fe640 1 -- 172.21.15.19:0/142621371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13f2390 msgr2=0x55beb1793810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197f7fe640 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13f2390 0x55beb1793810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197f7fe640 1 -- 172.21.15.19:0/142621371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb16bdc30 con 0x55beb13d16b0 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f19849bb640 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13f2390 0x55beb1793810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f19849bb640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19680afce0 0x7f19680b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197d7fa640 1 -- 172.21.15.19:0/2571259451 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f19740ff480 con 0x7f1980108a20 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f197f7fe640 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13d16b0 0x55beb17a06a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f19740fcb70 tx=0x7f19740f5da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9c86d6de43fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f19667fc640 1 -- 172.21.15.19:0/142621371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974049070 con 0x55beb13d16b0 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f19667fc640 1 -- 172.21.15.19:0/142621371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19740c7160 con 0x55beb13d16b0 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.672+0000 7f19667fc640 1 -- 172.21.15.19:0/142621371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740c7420 con 0x55beb13d16b0 2024-04-25T10:57:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 -- 172.21.15.19:0/142621371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13d16b0 msgr2=0x55beb17a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f19849bb640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19680afce0 0x7f19680b21f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55beb1790980 tx=0x7f1978004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13d16b0 0x55beb17a06a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f19740fcb70 tx=0x7f19740f5da0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 -- 172.21.15.19:0/142621371 shutdown_connections 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13d16b0 0x55beb17a06a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb1793d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 --2- 172.21.15.19:0/142621371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13f2390 0x55beb1793810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 -- 172.21.15.19:0/142621371 >> 172.21.15.19:0/142621371 conn(0x55beb13e64e0 msgr2=0x55beb178ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 -- 172.21.15.19:0/142621371 shutdown_connections 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.673+0000 7f198d3a8740 1 -- 172.21.15.19:0/142621371 wait complete. 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 -- start start 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 Processor -- start 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb179e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb179e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55beb16f8d80 con 0x55beb13f2390 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55beb167fdc0 con 0x55beb13d16b0 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55beb17a0be0 con 0x55beb13c94d0 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f198d3a8740 10 client.0 _ll_register_callbacks cb 0x55beb169b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f19849bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f19849bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43406/0 (socket says 172.21.15.19:43406) 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.674+0000 7f19849bb640 1 -- 172.21.15.19:0/351077464 learned_addr learned my addr 172.21.15.19:0/351077464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f197ffff640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb179e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f197f7fe640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb179e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f19849bb640 1 -- 172.21.15.19:0/351077464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 msgr2=0x55beb179e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f19849bb640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb179e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f19849bb640 1 -- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 msgr2=0x55beb179e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f19849bb640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb179e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f19849bb640 1 -- 172.21.15.19:0/351077464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb140a3f0 con 0x55beb13f2390 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f197ffff640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb179e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f19849bb640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a9fa0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1978012ef0 tx=0x7f1978004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb09bb7e0be966e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f195f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f195f7fe640 1 -- 172.21.15.19:0/351077464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978007de0 con 0x55beb13f2390 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19740c7160 con 0x55beb13f2390 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f195f7fe640 1 -- 172.21.15.19:0/351077464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f197801e450 con 0x55beb13f2390 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f195f7fe640 1 -- 172.21.15.19:0/351077464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1978027670 con 0x55beb13f2390 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f198d3a8740 10 client.15930 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.675+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55beb14d47d0 con 0x55beb13f2390 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f195f7fe640 1 -- 172.21.15.19:0/351077464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19740c7160 con 0x55beb13f2390 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f195f7fe640 1 -- 172.21.15.19:0/351077464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f197801ecb0 con 0x55beb13f2390 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f198d3a8740 10 client.15930 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f198d3a8740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f198d3a8740 20 client.15930 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f198d3a8740 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13b0f60 0x55beb17b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f198d3a8740 4 client.15930 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beb1706990 con 0x55beb13b0f60 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.676+0000 7f197ffff640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13b0f60 0x55beb17b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f195f7fe640 10 client.15930 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f197ffff640 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13b0f60 0x55beb17b03f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ef821da6d16fa6 server_cookie=421a9bb8b961b73c in_seq=0 out_seq=0 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f195f7fe640 1 -- 172.21.15.19:0/351077464 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55beb1706990 con 0x55beb13b0f60 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f195f7fe640 10 client.15930 handle_command_reply: tid=0 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 client.15930 shutdown 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 msgr2=0x55beb17a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a9fa0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1978012ef0 tx=0x7f1978004ab0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 shutdown_connections 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13b0f60 0x55beb17b03f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a9fa0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb179e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f195f7fe640 0 client.15930 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.677+0000 7f198d3a8740 1 --2- 172.21.15.19:0/351077464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb179e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.678+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 >> 172.21.15.19:0/351077464 conn(0x55beb13e64e0 msgr2=0x55beb1795530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.678+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 shutdown_connections 2024-04-25T10:57:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.678+0000 7f198d3a8740 1 -- 172.21.15.19:0/351077464 wait complete. 2024-04-25T10:57:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.678+0000 7f198d3a8740 20 client.15930 trim_cache size 0 max 16384 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.691+0000 7f198d3a8740 1 Processor -- start 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.691+0000 7f198d3a8740 1 -- start start 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.691+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb17a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.691+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb17b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.691+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f198d3a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19801ad8f0 con 0x55beb13f2390 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f198d3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b0170 con 0x55beb13d16b0 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f198d3a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19801adf60 con 0x55beb13c94d0 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43412/0 (socket says 172.21.15.19:43412) 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 -- 172.21.15.19:0/3222701394 learned_addr learned my addr 172.21.15.19:0/3222701394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f197ffff640 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb17a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f197f7fe640 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb17b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 -- 172.21.15.19:0/3222701394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 msgr2=0x55beb17a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb17a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 -- 172.21.15.19:0/3222701394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 msgr2=0x55beb17b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb17b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 -- 172.21.15.19:0/3222701394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb1794d90 con 0x55beb13f2390 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f197f7fe640 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb17b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f19849bb640 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17b6130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1978030450 tx=0x7f1978005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241eda48ba9edace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f1965ffb640 1 -- 172.21.15.19:0/3222701394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197803e070 con 0x55beb13f2390 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f1965ffb640 1 -- 172.21.15.19:0/3222701394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1978022070 con 0x55beb13f2390 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.692+0000 7f1965ffb640 1 -- 172.21.15.19:0/3222701394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978030e50 con 0x55beb13f2390 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 -- 172.21.15.19:0/3222701394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 msgr2=0x55beb17b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17b6130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1978030450 tx=0x7f1978005bd0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 -- 172.21.15.19:0/3222701394 shutdown_connections 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17b6130 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13d16b0 0x55beb17b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 --2- 172.21.15.19:0/3222701394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13c94d0 0x55beb17a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 -- 172.21.15.19:0/3222701394 >> 172.21.15.19:0/3222701394 conn(0x55beb13e64e0 msgr2=0x55beb17a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 -- 172.21.15.19:0/3222701394 shutdown_connections 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 -- 172.21.15.19:0/3222701394 wait complete. 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.693+0000 7f198d3a8740 1 -- start start 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 1 Processor -- start 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13c94d0 0x55beb17a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13d16b0 0x55beb17a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19801ae630 con 0x55beb13f2390 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19801afaa0 con 0x55beb13c94d0 2024-04-25T10:57:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19801aed00 con 0x55beb13d16b0 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f198d3a8740 10 client.0 _ll_register_callbacks cb 0x7f19780073a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f19849bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f19849bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43418/0 (socket says 172.21.15.19:43418) 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f19849bb640 1 -- 172.21.15.19:0/4047870858 learned_addr learned my addr 172.21.15.19:0/4047870858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f197ffff640 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13c94d0 0x55beb17a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f197f7fe640 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13d16b0 0x55beb17a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f19849bb640 1 -- 172.21.15.19:0/4047870858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13d16b0 msgr2=0x55beb17a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f19849bb640 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13d16b0 0x55beb17a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f19849bb640 1 -- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13c94d0 msgr2=0x55beb17a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.694+0000 7f19849bb640 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13c94d0 0x55beb17a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f19849bb640 1 -- 172.21.15.19:0/4047870858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb17a4420 con 0x55beb13f2390 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f19849bb640 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a3d30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f197803c040 tx=0x7f197801dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9ecf59f403b01c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f195f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f195f7fe640 1 -- 172.21.15.19:0/4047870858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978047060 con 0x55beb13f2390 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1978022070 con 0x55beb13f2390 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f195f7fe640 1 -- 172.21.15.19:0/4047870858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f1978030bd0 con 0x55beb13f2390 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f195f7fe640 1 -- 172.21.15.19:0/4047870858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f197803e040 con 0x55beb13f2390 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f198d3a8740 10 client.15936 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55beb18185c0 con 0x55beb13f2390 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f195f7fe640 1 -- 172.21.15.19:0/4047870858 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1978022070 con 0x55beb13f2390 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f195f7fe640 1 -- 172.21.15.19:0/4047870858 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1978005040 con 0x55beb13f2390 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f198d3a8740 10 client.15936 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f198d3a8740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.695+0000 7f198d3a8740 20 client.15936 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f198d3a8740 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13863a0 0x55beb181ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f198d3a8740 4 client.15936 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55beb1700800 con 0x55beb13863a0 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f197ffff640 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13863a0 0x55beb181ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f195f7fe640 10 client.15936 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f197ffff640 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13863a0 0x55beb181ac30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b81c1adf19e2ab80 server_cookie=9d739f2c2de2356a in_seq=0 out_seq=0 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f195f7fe640 1 -- 172.21.15.19:0/4047870858 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55beb1700800 con 0x55beb13863a0 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f195f7fe640 10 client.15936 handle_command_reply: tid=0 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.696+0000 7f198d3a8740 1 client.15936 shutdown 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 msgr2=0x55beb17a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a3d30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f197803c040 tx=0x7f197801dc40 comp rx=0 tx=0).stop 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 shutdown_connections 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55beb13863a0 0x55beb181ac30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb13f2390 0x55beb17a3d30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55beb13d16b0 0x55beb17a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f195f7fe640 0 client.15936 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 --2- 172.21.15.19:0/4047870858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55beb13c94d0 0x55beb17a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 >> 172.21.15.19:0/4047870858 conn(0x55beb13e64e0 msgr2=0x55beb17a8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 shutdown_connections 2024-04-25T10:57:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 1 -- 172.21.15.19:0/4047870858 wait complete. 2024-04-25T10:57:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198d3a8740 20 client.15936 trim_cache size 0 max 16384 2024-04-25T10:57:13.705 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:57:13.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19680afce0 msgr2=0x7f19680b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19680afce0 0x7f19680b21f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55beb1790980 tx=0x7f1978004a40 comp rx=0 tx=0).stop 2024-04-25T10:57:13.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108a20 msgr2=0x7f198019acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:13.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108a20 0x7f198019acf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1974032fd0 tx=0x7f1974004270 comp rx=0 tx=0).stop 2024-04-25T10:57:13.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 shutdown_connections 2024-04-25T10:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f19680afce0 0x7f19680b21f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1980108ec0 0x7f198019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980108a20 0x7f198019acf0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 --2- 172.21.15.19:0/2571259451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19801031d0 0x7f198019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 >> 172.21.15.19:0/2571259451 conn(0x7f1980078180 msgr2=0x7f1980074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.697+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 shutdown_connections 2024-04-25T10:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:13.698+0000 7f198641e640 1 -- 172.21.15.19:0/2571259451 wait complete. 2024-04-25T10:57:14.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:14 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/773406912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:14.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:14 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/773406912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:14.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:14 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/773406912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:14.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:57:14.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:57:14.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 -- 172.21.15.19:0/2889363241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90104270 msgr2=0x7ffb90110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 --2- 172.21.15.19:0/2889363241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90104270 0x7ffb90110640 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009ca0 tx=0x7ffb8802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 -- 172.21.15.19:0/2889363241 shutdown_connections 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 --2- 172.21.15.19:0/2889363241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90104270 0x7ffb90110640 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 --2- 172.21.15.19:0/2889363241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90109fc0 0x7ffb90103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 --2- 172.21.15.19:0/2889363241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb90109b20 0x7ffb901036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 -- 172.21.15.19:0/2889363241 >> 172.21.15.19:0/2889363241 conn(0x7ffb900ff1e0 msgr2=0x7ffb90101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 -- 172.21.15.19:0/2889363241 shutdown_connections 2024-04-25T10:57:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.840+0000 7ffb95092640 1 -- 172.21.15.19:0/2889363241 wait complete. 2024-04-25T10:57:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 Processor -- start 2024-04-25T10:57:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 -- start start 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb90104270 0x7ffb9019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90109b20 0x7ffb9019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90109fc0 0x7ffb9019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90113d30 con 0x7ffb90109b20 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90113bb0 con 0x7ffb90104270 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb95092640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90113eb0 con 0x7ffb90109fc0 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90109b20 0x7ffb9019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90109b20 0x7ffb9019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43440/0 (socket says 172.21.15.19:43440) 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb8ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb90104270 0x7ffb9019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb8e59c640 1 -- 172.21.15.19:0/210962825 learned_addr learned my addr 172.21.15.19:0/210962825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.841+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90109fc0 0x7ffb9019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb8e59c640 1 -- 172.21.15.19:0/210962825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90109fc0 msgr2=0x7ffb9019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90109fc0 0x7ffb9019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb8e59c640 1 -- 172.21.15.19:0/210962825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb90104270 msgr2=0x7ffb9019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb90104270 0x7ffb9019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb8e59c640 1 -- 172.21.15.19:0/210962825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9019f570 con 0x7ffb90109b20 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90109b20 0x7ffb9019ac70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005d00 tx=0x7ffb8400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb77fff640 1 -- 172.21.15.19:0/210962825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84017070 con 0x7ffb90109b20 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb77fff640 1 -- 172.21.15.19:0/210962825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb84009c30 con 0x7ffb90109b20 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9019f860 con 0x7ffb90109b20 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb77fff640 1 -- 172.21.15.19:0/210962825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400f620 con 0x7ffb90109b20 2024-04-25T10:57:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.842+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9019fd60 con 0x7ffb90109b20 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 Processor -- start 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 -- start start 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb77fff640 1 -- 172.21.15.19:0/210962825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ffb8401a070 con 0x7ffb90109b20 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea7854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea785f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557aea6ebb00 con 0x557aea3c4730 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557aea687920 con 0x557aea3bc550 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557aea6e7ac0 con 0x557aea3e5410 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea785f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea785f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43442/0 (socket says 172.21.15.19:43442) 2024-04-25T10:57:14.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb8e59c640 1 -- 172.21.15.19:0/1571918301 learned_addr learned my addr 172.21.15.19:0/1571918301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.845+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea7854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8e59c640 1 -- 172.21.15.19:0/1571918301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 msgr2=0x557aea7854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea7854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8e59c640 1 -- 172.21.15.19:0/1571918301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 msgr2=0x557aea7859f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7859f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8e59c640 1 -- 172.21.15.19:0/1571918301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aea4c57e0 con 0x557aea3c4730 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb77fff640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb780afc60 0x7ffb780b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea785f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb840dc090 tx=0x7ffb840e6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33aff03978c99570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb780afc60 0x7ffb780b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb757fa640 1 -- 172.21.15.19:0/1571918301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb840a2ab0 con 0x557aea3c4730 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb77fff640 1 -- 172.21.15.19:0/210962825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ffb840de970 con 0x7ffb90109b20 2024-04-25T10:57:14.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb757fa640 1 -- 172.21.15.19:0/1571918301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb840a6700 con 0x557aea3c4730 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.846+0000 7ffb757fa640 1 -- 172.21.15.19:0/1571918301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb840a6360 con 0x557aea3c4730 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1571918301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 msgr2=0x557aea785f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb780afc60 0x7ffb780b2170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557aea7827c0 tx=0x7ffb88009460 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea785f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb840dc090 tx=0x7ffb840e6670 comp rx=0 tx=0).stop 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1571918301 shutdown_connections 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea785f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1571918301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea7854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1571918301 >> 172.21.15.19:0/1571918301 conn(0x557aea3d9560 msgr2=0x557aea781dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1571918301 shutdown_connections 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.847+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1571918301 wait complete. 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 -- start start 2024-04-25T10:57:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 Processor -- start 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3bc550 0x557aea7940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3e5410 0x557aea79a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557aea796d20 con 0x557aea3bc550 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557aea6855d0 con 0x557aea3e5410 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557aea796ea0 con 0x557aea3c4730 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb9c01e740 10 client.0 _ll_register_callbacks cb 0x557aea673080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3bc550 0x557aea7940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3bc550 0x557aea7940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43452/0 (socket says 172.21.15.19:43452) 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 -- 172.21.15.19:0/1102405604 learned_addr learned my addr 172.21.15.19:0/1102405604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3e5410 0x557aea79a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 -- 172.21.15.19:0/1102405604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 msgr2=0x557aea7945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 -- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3e5410 msgr2=0x557aea79a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3e5410 0x557aea79a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8ed9d640 1 -- 172.21.15.19:0/1102405604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aea525f70 con 0x557aea3bc550 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.848+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3e5410 0x557aea79a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3bc550 0x557aea7940b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000bad0 tx=0x7ffb8000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5ef972b71d7837 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb72ff5640 1 -- 172.21.15.19:0/1102405604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80003e00 con 0x557aea3bc550 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb72ff5640 1 -- 172.21.15.19:0/1102405604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ffb80005020 con 0x557aea3bc550 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb840a6700 con 0x557aea3bc550 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb72ff5640 1 -- 172.21.15.19:0/1102405604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffb80005260 con 0x557aea3bc550 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb9c01e740 10 client.15948 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557aea3bd170 con 0x557aea3bc550 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb72ff5640 1 -- 172.21.15.19:0/1102405604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb840a6700 con 0x557aea3bc550 2024-04-25T10:57:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb72ff5640 1 -- 172.21.15.19:0/1102405604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ffb8001b600 con 0x557aea3bc550 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb9c01e740 10 client.15948 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb9c01e740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb9c01e740 20 client.15948 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.849+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea3a41f0 0x557aea7a6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.850+0000 7ffb9c01e740 4 client.15948 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.850+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aea6f9710 con 0x557aea3a41f0 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.850+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea3a41f0 0x557aea7a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.850+0000 7ffb72ff5640 10 client.15948 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.850+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea3a41f0 0x557aea7a6470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b5bfd3f1b54501 server_cookie=683ed6fdca2f90f3 in_seq=0 out_seq=0 2024-04-25T10:57:14.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb72ff5640 1 -- 172.21.15.19:0/1102405604 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557aea6f9710 con 0x557aea3a41f0 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb72ff5640 10 client.15948 handle_command_reply: tid=0 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 client.15948 shutdown 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3bc550 msgr2=0x557aea7940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3bc550 0x557aea7940b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000bad0 tx=0x7ffb8000bf90 comp rx=0 tx=0).stop 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 shutdown_connections 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea3a41f0 0x557aea7a6470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3e5410 0x557aea79a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/1102405604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3bc550 0x557aea7940b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb72ff5640 0 client.15948 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 >> 172.21.15.19:0/1102405604 conn(0x557aea3d9560 msgr2=0x557aea787340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 shutdown_connections 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 1 -- 172.21.15.19:0/1102405604 wait complete. 2024-04-25T10:57:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.851+0000 7ffb9c01e740 20 client.15948 trim_cache size 0 max 16384 2024-04-25T10:57:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.869+0000 7ffb9c01e740 1 Processor -- start 2024-04-25T10:57:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.869+0000 7ffb9c01e740 1 -- start start 2024-04-25T10:57:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea780ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea7879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea787ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901ab850 con 0x557aea3c4730 2024-04-25T10:57:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901ae0d0 con 0x557aea3bc550 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901abec0 con 0x557aea3e5410 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea7879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea7879b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43456/0 (socket says 172.21.15.19:43456) 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 -- 172.21.15.19:0/2290072864 learned_addr learned my addr 172.21.15.19:0/2290072864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea787ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea780ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 -- 172.21.15.19:0/2290072864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 msgr2=0x557aea787ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea787ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 -- 172.21.15.19:0/2290072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 msgr2=0x557aea780ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea780ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 -- 172.21.15.19:0/2290072864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aea786bb0 con 0x557aea3c4730 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea780ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea7879b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffb840dc6f0 tx=0x7ffb8408f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e75c14d1011a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:14.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb74ff9640 1 -- 172.21.15.19:0/2290072864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84021040 con 0x557aea3c4730 2024-04-25T10:57:14.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.870+0000 7ffb74ff9640 1 -- 172.21.15.19:0/2290072864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb840e4300 con 0x557aea3c4730 2024-04-25T10:57:14.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb74ff9640 1 -- 172.21.15.19:0/2290072864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb840a6540 con 0x557aea3c4730 2024-04-25T10:57:14.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 -- 172.21.15.19:0/2290072864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 msgr2=0x557aea7879b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea7879b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffb840dc6f0 tx=0x7ffb8408f040 comp rx=0 tx=0).stop 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 -- 172.21.15.19:0/2290072864 shutdown_connections 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3e5410 0x557aea787ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3c4730 0x557aea7879b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/2290072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea780ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 -- 172.21.15.19:0/2290072864 >> 172.21.15.19:0/2290072864 conn(0x557aea3d9560 msgr2=0x557aea7872a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 -- 172.21.15.19:0/2290072864 shutdown_connections 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.871+0000 7ffb9c01e740 1 -- 172.21.15.19:0/2290072864 wait complete. 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 -- start start 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 Processor -- start 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3e5410 0x557aea799aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901ac590 con 0x557aea3e5410 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901ada00 con 0x557aea3bc550 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901acc60 con 0x557aea3c4730 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb9c01e740 10 client.0 _ll_register_callbacks cb 0x7ffb80004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3e5410 0x557aea799aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3e5410 0x557aea799aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43460/0 (socket says 172.21.15.19:43460) 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8f59e640 1 -- 172.21.15.19:0/3740083003 learned_addr learned my addr 172.21.15.19:0/3740083003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8e59c640 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8f59e640 1 -- 172.21.15.19:0/3740083003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 msgr2=0x557aea7a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8f59e640 1 -- 172.21.15.19:0/3740083003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 msgr2=0x557aea7a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.872+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb8f59e640 1 -- 172.21.15.19:0/3740083003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aea79a190 con 0x557aea3e5410 2024-04-25T10:57:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb8f59e640 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3e5410 0x557aea799aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffb88035000 tx=0x7ffb880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e334572175d223c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb72ff5640 1 -- 172.21.15.19:0/3740083003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88059060 con 0x557aea3e5410 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb72ff5640 1 -- 172.21.15.19:0/3740083003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ffb88003e00 con 0x557aea3e5410 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb840e4300 con 0x557aea3e5410 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb72ff5640 1 -- 172.21.15.19:0/3740083003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffb88004040 con 0x557aea3e5410 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb9c01e740 10 client.15954 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557aea79eb30 con 0x557aea3e5410 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb72ff5640 1 -- 172.21.15.19:0/3740083003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb840e4300 con 0x557aea3e5410 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb72ff5640 1 -- 172.21.15.19:0/3740083003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ffb8802fc30 con 0x557aea3e5410 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.873+0000 7ffb9c01e740 10 client.15954 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb9c01e740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb9c01e740 20 client.15954 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:14.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea379630 0x557aea8108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb9c01e740 4 client.15954 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557aea6f3580 con 0x557aea379630 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea379630 0x557aea8108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb8ed9d640 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea379630 0x557aea8108e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6c927b64595526 server_cookie=d57cf6d6a87288fa in_seq=0 out_seq=0 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.874+0000 7ffb72ff5640 10 client.15954 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.875+0000 7ffb72ff5640 1 -- 172.21.15.19:0/3740083003 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557aea6f3580 con 0x557aea379630 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.875+0000 7ffb72ff5640 10 client.15954 handle_command_reply: tid=0 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.875+0000 7ffb9c01e740 1 client.15954 shutdown 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3e5410 msgr2=0x557aea799aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3e5410 0x557aea799aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffb88035000 tx=0x7ffb880095c0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 shutdown_connections 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557aea379630 0x557aea8108e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557aea3e5410 0x557aea799aa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557aea3c4730 0x557aea7a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb72ff5640 0 client.15954 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 --2- 172.21.15.19:0/3740083003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557aea3bc550 0x557aea7a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 >> 172.21.15.19:0/3740083003 conn(0x557aea3d9560 msgr2=0x557aea785de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 shutdown_connections 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 1 -- 172.21.15.19:0/3740083003 wait complete. 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb9c01e740 20 client.15954 trim_cache size 0 max 16384 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:57:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "124ca6f9-6fd9-4efe-88dc-919555f27182", 2024-04-25T10:57:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:57:14.883 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb780afc60 msgr2=0x7ffb780b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb780afc60 0x7ffb780b2170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557aea7827c0 tx=0x7ffb88009460 comp rx=0 tx=0).stop 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90109b20 msgr2=0x7ffb9019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90109b20 0x7ffb9019ac70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005d00 tx=0x7ffb8400dde0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 shutdown_connections 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffb780afc60 0x7ffb780b2170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90109fc0 0x7ffb9019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb90109b20 0x7ffb9019ac70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 --2- 172.21.15.19:0/210962825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb90104270 0x7ffb9019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.876+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 >> 172.21.15.19:0/210962825 conn(0x7ffb900ff1e0 msgr2=0x7ffb90101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:14.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.877+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 shutdown_connections 2024-04-25T10:57:14.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:14.877+0000 7ffb95092640 1 -- 172.21.15.19:0/210962825 wait complete. 2024-04-25T10:57:14.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:57:14.889 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:57:15.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 -- 172.21.15.19:0/718975448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e90108a30 msgr2=0x7f5e901025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 --2- 172.21.15.19:0/718975448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e90108a30 0x7f5e901025c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009900 tx=0x7f5e8802a410 comp rx=0 tx=0).stop 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 -- 172.21.15.19:0/718975448 shutdown_connections 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 --2- 172.21.15.19:0/718975448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e9010d390 0x7f5e9010f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 --2- 172.21.15.19:0/718975448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e90102b00 0x7f5e90102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 --2- 172.21.15.19:0/718975448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e90108a30 0x7f5e901025c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 -- 172.21.15.19:0/718975448 >> 172.21.15.19:0/718975448 conn(0x7f5e900fe260 msgr2=0x7f5e90100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 -- 172.21.15.19:0/718975448 shutdown_connections 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 -- 172.21.15.19:0/718975448 wait complete. 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 Processor -- start 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.003+0000 7f5e955ee640 1 -- start start 2024-04-25T10:57:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e955ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e90102b00 0x7f5e9019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e955ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e90108a30 0x7f5e9019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e955ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e9010d390 0x7f5e9019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e955ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90112bf0 con 0x7f5e9010d390 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e955ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90112a70 con 0x7f5e90102b00 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e955ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90112d70 con 0x7f5e90108a30 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e9010d390 0x7f5e9019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e9010d390 0x7f5e9019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43474/0 (socket says 172.21.15.19:43474) 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/3178931086 learned_addr learned my addr 172.21.15.19:0/3178931086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8effd640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e90102b00 0x7f5e9019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e90108a30 0x7f5e9019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/3178931086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e90108a30 msgr2=0x7f5e9019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e90108a30 0x7f5e9019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/3178931086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e90102b00 msgr2=0x7f5e9019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e90102b00 0x7f5e9019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/3178931086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9019f5e0 con 0x7f5e9010d390 2024-04-25T10:57:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.004+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e9010d390 0x7f5e9019efa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009fd0 tx=0x7f5e8000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.005+0000 7f5e77fff640 1 -- 172.21.15.19:0/3178931086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80017070 con 0x7f5e9010d390 2024-04-25T10:57:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.005+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9019f8d0 con 0x7f5e9010d390 2024-04-25T10:57:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.005+0000 7f5e77fff640 1 -- 172.21.15.19:0/3178931086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e80003e20 con 0x7f5e9010d390 2024-04-25T10:57:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.005+0000 7f5e77fff640 1 -- 172.21.15.19:0/3178931086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80010460 con 0x7f5e9010d390 2024-04-25T10:57:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.005+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9019fdd0 con 0x7f5e9010d390 2024-04-25T10:57:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.007+0000 7f5e9c577740 1 Processor -- start 2024-04-25T10:57:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.007+0000 7f5e77fff640 1 -- 172.21.15.19:0/3178931086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5e80003970 con 0x7f5e9010d390 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.007+0000 7f5e9c577740 1 -- start start 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c874a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c874fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4b36b0 0x55989c8818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e9c577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55989c7dad80 con 0x55989c4d4390 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e9c577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55989c7948b0 con 0x55989c4ab4d0 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e9c577740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55989c7d6d40 con 0x55989c4b36b0 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c874a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c874a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43490/0 (socket says 172.21.15.19:43490) 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/1809982601 learned_addr learned my addr 172.21.15.19:0/1809982601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8effd640 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c874fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/1809982601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4b36b0 msgr2=0x55989c8818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4b36b0 0x55989c8818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/1809982601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 msgr2=0x55989c874fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c874fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/1809982601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989c765a40 con 0x55989c4d4390 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e77fff640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f5e780afc60 0x7f5e780b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8effd640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f5e780afc60 0x7f5e780b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e77fff640 1 -- 172.21.15.19:0/3178931086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f5e800dee90 con 0x7f5e9010d390 2024-04-25T10:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.008+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c874a60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009c70 tx=0x7f5e84012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39141e6b4b628b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.009+0000 7f5e757fa640 1 -- 172.21.15.19:0/1809982601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84005020 con 0x55989c4d4390 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.009+0000 7f5e757fa640 1 -- 172.21.15.19:0/1809982601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e840047a0 con 0x55989c4d4390 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.009+0000 7f5e757fa640 1 -- 172.21.15.19:0/1809982601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84013660 con 0x55989c4d4390 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.009+0000 7f5e9c577740 1 -- 172.21.15.19:0/1809982601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 msgr2=0x55989c874a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.009+0000 7f5e8effd640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f5e780afc60 0x7f5e780b2170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c00 tx=0x7f5e88005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.009+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c874a60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009c70 tx=0x7f5e84012760 comp rx=0 tx=0).stop 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- 172.21.15.19:0/1809982601 shutdown_connections 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4b36b0 0x55989c8818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c874fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1809982601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c874a60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- 172.21.15.19:0/1809982601 >> 172.21.15.19:0/1809982601 conn(0x55989c4c84e0 msgr2=0x55989c871210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- 172.21.15.19:0/1809982601 shutdown_connections 2024-04-25T10:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- 172.21.15.19:0/1809982601 wait complete. 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- start start 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 Processor -- start 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c87f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4b36b0 0x55989c87fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4d4390 0x55989c885430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55989c881fe0 con 0x55989c4b36b0 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55989c761d10 con 0x55989c4ab4d0 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55989c882160 con 0x55989c4d4390 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e9c577740 10 client.0 _ll_register_callbacks cb 0x55989c78e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.010+0000 7f5e8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4b36b0 0x55989c87fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4b36b0 0x55989c87fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43504/0 (socket says 172.21.15.19:43504) 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/3522740664 learned_addr learned my addr 172.21.15.19:0/3522740664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8effd640 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c87f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4d4390 0x55989c885430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/3522740664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4d4390 msgr2=0x55989c885430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4d4390 0x55989c885430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 msgr2=0x55989c87f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c87f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 -- 172.21.15.19:0/3522740664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989c5b67d0 con 0x55989c4b36b0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4b36b0 0x55989c87fb30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5e84004e60 tx=0x7f5e84005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaa9ee804a51141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e72ff5640 1 -- 172.21.15.19:0/3522740664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84005430 con 0x55989c4b36b0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e72ff5640 1 -- 172.21.15.19:0/3522740664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f5e84004020 con 0x55989c4b36b0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e840047a0 con 0x55989c4b36b0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e72ff5640 1 -- 172.21.15.19:0/3522740664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5e84004220 con 0x55989c4b36b0 2024-04-25T10:57:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e9c577740 10 client.15966 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.011+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55989c4ec3f0 con 0x55989c4b36b0 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e72ff5640 1 -- 172.21.15.19:0/3522740664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e840047a0 con 0x55989c4b36b0 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e72ff5640 1 -- 172.21.15.19:0/3522740664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f5e8400f040 con 0x55989c4b36b0 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e9c577740 10 client.15966 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e9c577740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e9c577740 20 client.15966 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e9c577740 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c492f60 0x55989c8915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e9c577740 4 client.15966 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55989c7e8990 con 0x55989c492f60 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.012+0000 7f5e8effd640 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c492f60 0x55989c8915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e8effd640 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c492f60 0x55989c8915d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d768ead974849cd1 server_cookie=3dad1302e4c2ca6e in_seq=0 out_seq=0 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e72ff5640 10 client.15966 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e72ff5640 1 -- 172.21.15.19:0/3522740664 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55989c7e8990 con 0x55989c492f60 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e72ff5640 10 client.15966 handle_command_reply: tid=0 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 client.15966 shutdown 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4b36b0 msgr2=0x55989c87fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4b36b0 0x55989c87fb30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5e84004e60 tx=0x7f5e84005850 comp rx=0 tx=0).stop 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 shutdown_connections 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c492f60 0x55989c8915d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4d4390 0x55989c885430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4b36b0 0x55989c87fb30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e72ff5640 0 client.15966 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 --2- 172.21.15.19:0/3522740664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c87f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.013+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 >> 172.21.15.19:0/3522740664 conn(0x55989c4c84e0 msgr2=0x55989c876780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.014+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 shutdown_connections 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.014+0000 7f5e9c577740 1 -- 172.21.15.19:0/3522740664 wait complete. 2024-04-25T10:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.014+0000 7f5e9c577740 20 client.15966 trim_cache size 0 max 16384 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 Processor -- start 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 -- start start 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c492f60 0x55989c877170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4ab4d0 0x55989c874e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4d4390 0x55989c871220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e901ad900 con 0x55989c492f60 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e901b0180 con 0x55989c4d4390 2024-04-25T10:57:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e9c577740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e901adf70 con 0x55989c4ab4d0 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c492f60 0x55989c877170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c492f60 0x55989c877170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43506/0 (socket says 172.21.15.19:43506) 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 -- 172.21.15.19:0/781687108 learned_addr learned my addr 172.21.15.19:0/781687108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4d4390 0x55989c871220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4ab4d0 0x55989c874e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 -- 172.21.15.19:0/781687108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4ab4d0 msgr2=0x55989c874e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4ab4d0 0x55989c874e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 -- 172.21.15.19:0/781687108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4d4390 msgr2=0x55989c871220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4d4390 0x55989c871220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.032+0000 7f5e8effd640 1 -- 172.21.15.19:0/781687108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989c4ac0f0 con 0x55989c492f60 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.033+0000 7f5e8effd640 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c492f60 0x55989c877170 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e88005ab0 tx=0x7f5e8803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85a7349f326eb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.033+0000 7f5e74ff9640 1 -- 172.21.15.19:0/781687108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88057070 con 0x55989c492f60 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.033+0000 7f5e74ff9640 1 -- 172.21.15.19:0/781687108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e88005040 con 0x55989c492f60 2024-04-25T10:57:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.033+0000 7f5e74ff9640 1 -- 172.21.15.19:0/781687108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88035040 con 0x55989c492f60 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- 172.21.15.19:0/781687108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c492f60 msgr2=0x55989c877170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c492f60 0x55989c877170 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e88005ab0 tx=0x7f5e8803e000 comp rx=0 tx=0).stop 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- 172.21.15.19:0/781687108 shutdown_connections 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4d4390 0x55989c871220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c4ab4d0 0x55989c874e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 --2- 172.21.15.19:0/781687108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c492f60 0x55989c877170 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- 172.21.15.19:0/781687108 >> 172.21.15.19:0/781687108 conn(0x55989c4c84e0 msgr2=0x55989c880b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- 172.21.15.19:0/781687108 shutdown_connections 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- 172.21.15.19:0/781687108 wait complete. 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- start start 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 Processor -- start 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c492f60 0x55989c886400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c886940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c886e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e901ae640 con 0x55989c4d4390 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e901afab0 con 0x55989c4ab4d0 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e901aed10 con 0x55989c492f60 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e9c577740 10 client.0 _ll_register_callbacks cb 0x7f5e84005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.034+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c886e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c886e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43520/0 (socket says 172.21.15.19:43520) 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/1732603732 learned_addr learned my addr 172.21.15.19:0/1732603732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8e7fc640 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c886940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8effd640 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c492f60 0x55989c886400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/1732603732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c492f60 msgr2=0x55989c886400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c492f60 0x55989c886400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 msgr2=0x55989c886940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c886940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 -- 172.21.15.19:0/1732603732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989c79b4b0 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e8f7fe640 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c886e80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5e800dcc10 tx=0x7f5e800dcb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5edc4a5a25e3fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e72ff5640 1 -- 172.21.15.19:0/1732603732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80020040 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e88005040 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e72ff5640 1 -- 172.21.15.19:0/1732603732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f5e800ab0d0 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e72ff5640 1 -- 172.21.15.19:0/1732603732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5e800a7060 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e9c577740 10 client.15972 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55989c8fce80 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.035+0000 7f5e72ff5640 1 -- 172.21.15.19:0/1732603732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e88005040 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e72ff5640 1 -- 172.21.15.19:0/1732603732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f5e800a5d30 con 0x55989c4d4390 2024-04-25T10:57:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e9c577740 10 client.15972 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e9c577740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e9c577740 20 client.15972 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c4683a0 0x55989c8ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e9c577740 4 client.15972 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55989c7e2800 con 0x55989c4683a0 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e8effd640 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c4683a0 0x55989c8ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e8effd640 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c4683a0 0x55989c8ff5c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=661b903e1ec91ce7 server_cookie=df1b6ac885952e72 in_seq=0 out_seq=0 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.036+0000 7f5e72ff5640 10 client.15972 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e72ff5640 1 -- 172.21.15.19:0/1732603732 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55989c7e2800 con 0x55989c4683a0 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e72ff5640 10 client.15972 handle_command_reply: tid=0 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 client.15972 shutdown 2024-04-25T10:57:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 msgr2=0x55989c886e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c886e80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5e800dcc10 tx=0x7f5e800dcb10 comp rx=0 tx=0).stop 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 shutdown_connections 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55989c4683a0 0x55989c8ff5c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55989c4d4390 0x55989c886e80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e72ff5640 0 client.15972 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55989c4ab4d0 0x55989c886940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 --2- 172.21.15.19:0/1732603732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55989c492f60 0x55989c886400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 >> 172.21.15.19:0/1732603732 conn(0x55989c4c84e0 msgr2=0x55989c870150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 shutdown_connections 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 1 -- 172.21.15.19:0/1732603732 wait complete. 2024-04-25T10:57:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e9c577740 20 client.15972 trim_cache size 0 max 16384 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "124ca6f9-6fd9-4efe-88dc-919555f27182": { 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "124ca6f9-6fd9-4efe-88dc-919555f27182", 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:57:15.046 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:15.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f5e780afc60 msgr2=0x7f5e780b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e955ee640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f5e780afc60 0x7f5e780b2170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c00 tx=0x7f5e88005a20 comp rx=0 tx=0).stop 2024-04-25T10:57:15.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e9010d390 msgr2=0x7f5e9019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:15.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.037+0000 7f5e955ee640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e9010d390 0x7f5e9019efa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009fd0 tx=0x7f5e8000b260 comp rx=0 tx=0).stop 2024-04-25T10:57:15.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 shutdown_connections 2024-04-25T10:57:15.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f5e780afc60 0x7f5e780b2170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e9010d390 0x7f5e9019efa0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e90108a30 0x7f5e9019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 --2- 172.21.15.19:0/3178931086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e90102b00 0x7f5e9019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 >> 172.21.15.19:0/3178931086 conn(0x7f5e900fe260 msgr2=0x7f5e90100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 shutdown_connections 2024-04-25T10:57:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:15.038+0000 7f5e955ee640 1 -- 172.21.15.19:0/3178931086 wait complete. 2024-04-25T10:57:15.050 INFO:tasks.cephfs.filesystem:scrub status for tag:124ca6f9-6fd9-4efe-88dc-919555f27182 - {'path': '/', 'tag': '124ca6f9-6fd9-4efe-88dc-919555f27182', 'options': 'recursive,force'} 2024-04-25T10:57:15.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:15 smithi019 ceph-mon[29209]: pgmap v85: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T10:57:15.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:15 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:57:15.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:15 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:15.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:15 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:57:15.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:15 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:15 smithi169 ceph-mon[33049]: pgmap v85: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T10:57:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:15 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:57:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:15 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:15 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:57:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:15 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:15.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:15 smithi112 ceph-mon[32799]: pgmap v85: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T10:57:15.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:15 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:57:15.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:15 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:15.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:15 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:57:15.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:15 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:17 smithi019 ceph-mon[29209]: pgmap v86: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 3 op/s 2024-04-25T10:57:17.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:17 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:57:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:17 smithi169 ceph-mon[33049]: pgmap v86: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 3 op/s 2024-04-25T10:57:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:17 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:57:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:17 smithi112 ceph-mon[32799]: pgmap v86: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 3 op/s 2024-04-25T10:57:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:17 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:57:19.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:19 smithi019 ceph-mon[29209]: pgmap v87: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:57:19.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:19 smithi169 ceph-mon[33049]: pgmap v87: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:57:19.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:19 smithi112 ceph-mon[32799]: pgmap v87: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:57:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:21 smithi019 ceph-mon[29209]: pgmap v88: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:57:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:21 smithi169 ceph-mon[33049]: pgmap v88: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:57:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:21 smithi112 ceph-mon[32799]: pgmap v88: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:57:23.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:23 smithi019 ceph-mon[29209]: pgmap v89: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T10:57:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:23 smithi169 ceph-mon[33049]: pgmap v89: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T10:57:23.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:23 smithi112 ceph-mon[32799]: pgmap v89: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T10:57:25.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:25 smithi019 ceph-mon[29209]: pgmap v90: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:57:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:25 smithi169 ceph-mon[33049]: pgmap v90: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:57:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:25 smithi112 ceph-mon[32799]: pgmap v90: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:57:27.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:27 smithi019 ceph-mon[29209]: pgmap v91: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:57:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:27 smithi169 ceph-mon[33049]: pgmap v91: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:57:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:27 smithi112 ceph-mon[32799]: pgmap v91: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:57:29.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:29 smithi019 ceph-mon[29209]: pgmap v92: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:29.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:29 smithi169 ceph-mon[33049]: pgmap v92: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:29.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:29 smithi112 ceph-mon[32799]: pgmap v92: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:31 smithi169 ceph-mon[33049]: pgmap v93: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:31.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:31 smithi112 ceph-mon[32799]: pgmap v93: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:31.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:31 smithi019 ceph-mon[29209]: pgmap v93: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:33 smithi169 ceph-mon[33049]: pgmap v94: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:33.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:33 smithi112 ceph-mon[32799]: pgmap v94: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:33.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:33 smithi019 ceph-mon[29209]: pgmap v94: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:35.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:35 smithi169 ceph-mon[33049]: pgmap v95: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:35.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:35 smithi112 ceph-mon[32799]: pgmap v95: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:35.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:35 smithi019 ceph-mon[29209]: pgmap v95: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:37 smithi169 ceph-mon[33049]: pgmap v96: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:37 smithi112 ceph-mon[32799]: pgmap v96: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:37.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:37 smithi019 ceph-mon[29209]: pgmap v96: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:39 smithi169 ceph-mon[33049]: pgmap v97: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:39 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:57:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:39 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:57:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:39 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:57:39.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:39 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:57:39.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:39 smithi112 ceph-mon[32799]: pgmap v97: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:39 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:57:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:39 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:57:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:39 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:57:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:39 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:57:39.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:39 smithi019 ceph-mon[29209]: pgmap v97: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:39 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:57:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:39 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:57:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:39 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:57:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:39 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:57:41.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:41 smithi169 ceph-mon[33049]: pgmap v98: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:41.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:41 smithi112 ceph-mon[32799]: pgmap v98: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:41.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:41 smithi019 ceph-mon[29209]: pgmap v98: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:43 smithi169 ceph-mon[33049]: pgmap v99: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:43 smithi112 ceph-mon[32799]: pgmap v99: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:43.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:43 smithi019 ceph-mon[29209]: pgmap v99: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:45.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:57:45.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 -- 172.21.15.19:0/3684550060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 msgr2=0x7f5848110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 --2- 172.21.15.19:0/3684550060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f5848110570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5840009900 tx=0x7f584002a410 comp rx=0 tx=0).stop 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 -- 172.21.15.19:0/3684550060 shutdown_connections 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 --2- 172.21.15.19:0/3684550060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f5848110570 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 --2- 172.21.15.19:0/3684550060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f584810b690 0x7f584810dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 --2- 172.21.15.19:0/3684550060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848104c30 0x7f5848105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 -- 172.21.15.19:0/3684550060 >> 172.21.15.19:0/3684550060 conn(0x7f5848078110 msgr2=0x7f5848078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 -- 172.21.15.19:0/3684550060 shutdown_connections 2024-04-25T10:57:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.182+0000 7f584d5b7640 1 -- 172.21.15.19:0/3684550060 wait complete. 2024-04-25T10:57:45.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 Processor -- start 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 -- start start 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848104c30 0x7f584819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f584810b690 0x7f584819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f584819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5848113ce0 con 0x7f584810e080 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5848113b60 con 0x7f584810b690 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f584d5b7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5848113e60 con 0x7f5848104c30 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f584819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f584819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34932/0 (socket says 172.21.15.19:34932) 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f58477fe640 1 -- 172.21.15.19:0/4119669209 learned_addr learned my addr 172.21.15.19:0/4119669209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.183+0000 7f5846ffd640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848104c30 0x7f584819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58467fc640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f584810b690 0x7f584819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58477fe640 1 -- 172.21.15.19:0/4119669209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848104c30 msgr2=0x7f584819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58477fe640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848104c30 0x7f584819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58477fe640 1 -- 172.21.15.19:0/4119669209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f584810b690 msgr2=0x7f584819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58477fe640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f584810b690 0x7f584819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58477fe640 1 -- 172.21.15.19:0/4119669209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584819f520 con 0x7f584810e080 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58467fc640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f584810b690 0x7f584819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f58477fe640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f584819eee0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5840004770 tx=0x7f5840004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f582ffff640 1 -- 172.21.15.19:0/4119669209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584003f030 con 0x7f584810e080 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584819f7b0 con 0x7f584810e080 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f582ffff640 1 -- 172.21.15.19:0/4119669209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5840003aa0 con 0x7f584810e080 2024-04-25T10:57:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f582ffff640 1 -- 172.21.15.19:0/4119669209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840035070 con 0x7f584810e080 2024-04-25T10:57:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.184+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584819fcf0 con 0x7f584810e080 2024-04-25T10:57:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 Processor -- start 2024-04-25T10:57:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f582ffff640 1 -- 172.21.15.19:0/4119669209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5840042030 con 0x7f584810e080 2024-04-25T10:57:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 -- start start 2024-04-25T10:57:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e193a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e193fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562f7e0f9d80 con 0x562f7ddf3390 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f7e0b38b0 con 0x562f7ddca4d0 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.187+0000 7f585446c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562f7e0f5d40 con 0x562f7ddd26b0 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e193a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e193a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34936/0 (socket says 172.21.15.19:34936) 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f5846ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e193fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 -- 172.21.15.19:0/3677765582 learned_addr learned my addr 172.21.15.19:0/3677765582 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 -- 172.21.15.19:0/3677765582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 msgr2=0x562f7e1a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 -- 172.21.15.19:0/3677765582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 msgr2=0x562f7e193fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e193fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 -- 172.21.15.19:0/3677765582 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7e084a40 con 0x562f7ddf3390 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f582ffff640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f58300afc60 0x7f58300b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f5846ffd640 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e193fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f5846ffd640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f58300afc60 0x7f58300b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f58467fc640 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e193a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f583c00bf20 tx=0x7f583c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59481fe9b762a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f582ffff640 1 -- 172.21.15.19:0/4119669209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f58400ff600 con 0x7f584810e080 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.188+0000 7f582d7fa640 1 -- 172.21.15.19:0/3677765582 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c004030 con 0x562f7ddf3390 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f582d7fa640 1 -- 172.21.15.19:0/3677765582 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c019070 con 0x562f7ddf3390 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f582d7fa640 1 -- 172.21.15.19:0/3677765582 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c00f3f0 con 0x562f7ddf3390 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 -- 172.21.15.19:0/3677765582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 msgr2=0x562f7e193a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e193a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f583c00bf20 tx=0x7f583c00b6b0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f5846ffd640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f58300afc60 0x7f58300b2170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x562f7e195230 tx=0x7f5838009760 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 -- 172.21.15.19:0/3677765582 shutdown_connections 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e193fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 --2- 172.21.15.19:0/3677765582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e193a60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 -- 172.21.15.19:0/3677765582 >> 172.21.15.19:0/3677765582 conn(0x562f7dde74e0 msgr2=0x562f7e190210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 -- 172.21.15.19:0/3677765582 shutdown_connections 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.189+0000 7f585446c740 1 -- 172.21.15.19:0/3677765582 wait complete. 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 -- start start 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 Processor -- start 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562f7e1a0fe0 con 0x562f7ddf3390 2024-04-25T10:57:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f7e080d10 con 0x562f7ddca4d0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562f7e1a1160 con 0x562f7ddd26b0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f585446c740 10 client.0 _ll_register_callbacks cb 0x562f7e0ad210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.190+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34950/0 (socket says 172.21.15.19:34950) 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 -- 172.21.15.19:0/500912060 learned_addr learned my addr 172.21.15.19:0/500912060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58467fc640 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f5846ffd640 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 -- 172.21.15.19:0/500912060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 msgr2=0x562f7e1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 -- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 msgr2=0x562f7e1a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 -- 172.21.15.19:0/500912060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7ded57d0 con 0x562f7ddf3390 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f58477fe640 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e19e4d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f58400c3850 tx=0x7f58400c7660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6498e547518aa07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f582aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:45.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f582aff5640 1 -- 172.21.15.19:0/500912060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840049070 con 0x562f7ddf3390 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f582aff5640 1 -- 172.21.15.19:0/500912060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f58400fd000 con 0x562f7ddf3390 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f583c019070 con 0x562f7ddf3390 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f582aff5640 1 -- 172.21.15.19:0/500912060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f58400c7ed0 con 0x562f7ddf3390 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f585446c740 10 client.15984 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.191+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562f7de0b3f0 con 0x562f7ddf3390 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f582aff5640 1 -- 172.21.15.19:0/500912060 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c019070 con 0x562f7ddf3390 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f582aff5640 1 -- 172.21.15.19:0/500912060 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f58400b00b0 con 0x562f7ddf3390 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f585446c740 10 client.15984 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f585446c740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f585446c740 20 client.15984 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f585446c740 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7ddb1f60 0x562f7e1b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f585446c740 4 client.15984 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f7e107990 con 0x562f7ddb1f60 2024-04-25T10:57:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.192+0000 7f5846ffd640 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7ddb1f60 0x562f7e1b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.193+0000 7f582aff5640 10 client.15984 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.193+0000 7f5846ffd640 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7ddb1f60 0x562f7e1b06c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714f8116f00bd21f server_cookie=bd6a733439bbb89 in_seq=0 out_seq=0 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.193+0000 7f582aff5640 1 -- 172.21.15.19:0/500912060 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562f7e107990 con 0x562f7ddb1f60 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.193+0000 7f582aff5640 10 client.15984 handle_command_reply: tid=0 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 client.15984 shutdown 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 msgr2=0x562f7e19e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e19e4d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f58400c3850 tx=0x7f58400c7660 comp rx=0 tx=0).stop 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 shutdown_connections 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7ddb1f60 0x562f7e1b06c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e19e4d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f582aff5640 0 client.15984 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 --2- 172.21.15.19:0/500912060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 >> 172.21.15.19:0/500912060 conn(0x562f7dde74e0 msgr2=0x562f7e18e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 shutdown_connections 2024-04-25T10:57:45.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 1 -- 172.21.15.19:0/500912060 wait complete. 2024-04-25T10:57:45.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.194+0000 7f585446c740 20 client.15984 trim_cache size 0 max 16384 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 Processor -- start 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 -- start start 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e19d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e18f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58481ad870 con 0x562f7ddf3390 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58481b00f0 con 0x562f7ddca4d0 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.212+0000 7f585446c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58481adee0 con 0x562f7ddd26b0 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34964/0 (socket says 172.21.15.19:34964) 2024-04-25T10:57:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 -- 172.21.15.19:0/2000648587 learned_addr learned my addr 172.21.15.19:0/2000648587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f5846ffd640 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58467fc640 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e18f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 -- 172.21.15.19:0/2000648587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 msgr2=0x562f7e18f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e18f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 -- 172.21.15.19:0/2000648587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 msgr2=0x562f7e19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 -- 172.21.15.19:0/2000648587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7ddcb0f0 con 0x562f7ddf3390 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f58477fe640 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1996d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f58400c7760 tx=0x7f58400c76d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00a01b701c7e5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f582cff9640 1 -- 172.21.15.19:0/2000648587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840049070 con 0x562f7ddf3390 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f582cff9640 1 -- 172.21.15.19:0/2000648587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58400c77d0 con 0x562f7ddf3390 2024-04-25T10:57:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.213+0000 7f582cff9640 1 -- 172.21.15.19:0/2000648587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58400fd000 con 0x562f7ddf3390 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 -- 172.21.15.19:0/2000648587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 msgr2=0x562f7e1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1996d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f58400c7760 tx=0x7f58400c76d0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 -- 172.21.15.19:0/2000648587 shutdown_connections 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1996d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e18f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 --2- 172.21.15.19:0/2000648587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 -- 172.21.15.19:0/2000648587 >> 172.21.15.19:0/2000648587 conn(0x562f7dde74e0 msgr2=0x562f7e18e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 -- 172.21.15.19:0/2000648587 shutdown_connections 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 -- 172.21.15.19:0/2000648587 wait complete. 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 -- start start 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.214+0000 7f585446c740 1 Processor -- start 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f585446c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f585446c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f585446c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f585446c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58481ae5b0 con 0x562f7ddf3390 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f585446c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58481afa20 con 0x562f7ddca4d0 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f585446c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58481aec80 con 0x562f7ddd26b0 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f585446c740 10 client.0 _ll_register_callbacks cb 0x7f58400c3690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34970/0 (socket says 172.21.15.19:34970) 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 -- 172.21.15.19:0/1911122389 learned_addr learned my addr 172.21.15.19:0/1911122389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f5846ffd640 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58467fc640 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 -- 172.21.15.19:0/1911122389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 msgr2=0x562f7e1a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 -- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 msgr2=0x562f7e1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 -- 172.21.15.19:0/1911122389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7e0ba4b0 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f58477fe640 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1a41b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5840032fd0 tx=0x7f5840003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c60358c6f3ded1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f582aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.215+0000 7f582aff5640 1 -- 172.21.15.19:0/1911122389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840049070 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f582aff5640 1 -- 172.21.15.19:0/1911122389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f58400c7a30 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58400c77d0 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f582aff5640 1 -- 172.21.15.19:0/1911122389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f58400c7cb0 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 10 client.15990 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562f7e0ba4b0 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f582aff5640 1 -- 172.21.15.19:0/1911122389 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58400c77d0 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f582aff5640 1 -- 172.21.15.19:0/1911122389 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f58400c6ac0 con 0x562f7ddf3390 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 10 client.15990 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 20 client.15990 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7dd873a0 0x562f7e21edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 4 client.15990 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f7e101800 con 0x562f7dd873a0 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.216+0000 7f5846ffd640 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7dd873a0 0x562f7e21edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f5846ffd640 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7dd873a0 0x562f7e21edc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69afa9ac709c0604 server_cookie=200cceecd1977a09 in_seq=0 out_seq=0 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f582aff5640 10 client.15990 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f582aff5640 1 -- 172.21.15.19:0/1911122389 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562f7e101800 con 0x562f7dd873a0 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f582aff5640 10 client.15990 handle_command_reply: tid=0 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 client.15990 shutdown 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 msgr2=0x562f7e1a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1a41b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5840032fd0 tx=0x7f5840003f50 comp rx=0 tx=0).stop 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 shutdown_connections 2024-04-25T10:57:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562f7dd873a0 0x562f7e21edc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562f7ddf3390 0x562f7e1a41b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f7ddd26b0 0x562f7e1a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f582aff5640 0 client.15990 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 --2- 172.21.15.19:0/1911122389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f7ddca4d0 0x562f7e1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 >> 172.21.15.19:0/1911122389 conn(0x562f7dde74e0 msgr2=0x562f7e18f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 shutdown_connections 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 1 -- 172.21.15.19:0/1911122389 wait complete. 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.217+0000 7f585446c740 20 client.15990 trim_cache size 0 max 16384 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:57:45.224 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:57:45.225 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f58300afc60 msgr2=0x7f58300b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f58300afc60 0x7f58300b2170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x562f7e195230 tx=0x7f5838009760 comp rx=0 tx=0).stop 2024-04-25T10:57:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 msgr2=0x7f584819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f584819eee0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5840004770 tx=0x7f5840004270 comp rx=0 tx=0).stop 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 shutdown_connections 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f58300afc60 0x7f58300b2170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584810e080 0x7f584819eee0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f584810b690 0x7f584819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 --2- 172.21.15.19:0/4119669209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5848104c30 0x7f584819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 >> 172.21.15.19:0/4119669209 conn(0x7f5848078110 msgr2=0x7f5848101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 shutdown_connections 2024-04-25T10:57:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.218+0000 7f584d5b7640 1 -- 172.21.15.19:0/4119669209 wait complete. 2024-04-25T10:57:45.230 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:57:45.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:57:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 -- 172.21.15.19:0/1974637456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10e040 msgr2=0x7fec9c110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 --2- 172.21.15.19:0/1974637456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10e040 0x7fec9c110530 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fec94009900 tx=0x7fec9402a410 comp rx=0 tx=0).stop 2024-04-25T10:57:45.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 -- 172.21.15.19:0/1974637456 shutdown_connections 2024-04-25T10:57:45.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 --2- 172.21.15.19:0/1974637456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10e040 0x7fec9c110530 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 --2- 172.21.15.19:0/1974637456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9c10b650 0x7fec9c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 --2- 172.21.15.19:0/1974637456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec9c104bf0 0x7fec9c105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 -- 172.21.15.19:0/1974637456 >> 172.21.15.19:0/1974637456 conn(0x7fec9c078110 msgr2=0x7fec9c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 -- 172.21.15.19:0/1974637456 shutdown_connections 2024-04-25T10:57:45.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.342+0000 7feca4306640 1 -- 172.21.15.19:0/1974637456 wait complete. 2024-04-25T10:57:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 Processor -- start 2024-04-25T10:57:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 -- start start 2024-04-25T10:57:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec9c104bf0 0x7fec9c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10b650 0x7fec9c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9c10e040 0x7fec9c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c113ca0 con 0x7fec9c10b650 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c113b20 con 0x7fec9c104bf0 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca4306640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c113e20 con 0x7fec9c10e040 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10b650 0x7fec9c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10b650 0x7fec9c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34992/0 (socket says 172.21.15.19:34992) 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 -- 172.21.15.19:0/1343527663 learned_addr learned my addr 172.21.15.19:0/1343527663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca20a2640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec9c104bf0 0x7fec9c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca28a3640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9c10e040 0x7fec9c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 -- 172.21.15.19:0/1343527663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9c10e040 msgr2=0x7fec9c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9c10e040 0x7fec9c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 -- 172.21.15.19:0/1343527663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec9c104bf0 msgr2=0x7fec9c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec9c104bf0 0x7fec9c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.343+0000 7feca18a1640 1 -- 172.21.15.19:0/1343527663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c19f4a0 con 0x7fec9c10b650 2024-04-25T10:57:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.344+0000 7feca18a1640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10b650 0x7fec9c19aba0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d950 tx=0x7fec8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.344+0000 7fec937fe640 1 -- 172.21.15.19:0/1343527663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c017070 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.344+0000 7fec937fe640 1 -- 172.21.15.19:0/1343527663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec8c009c30 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.344+0000 7fec937fe640 1 -- 172.21.15.19:0/1343527663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00f620 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.344+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c19f790 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.344+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c1a7110 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.345+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9c06a770 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.346+0000 7fec937fe640 1 -- 172.21.15.19:0/1343527663 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fec8c010460 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.347+0000 7fec937fe640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fec880afc60 0x7fec880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.347+0000 7fec937fe640 1 -- 172.21.15.19:0/1343527663 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fec8c0deaf0 con 0x7fec9c10b650 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.347+0000 7feca20a2640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fec880afc60 0x7fec880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.348+0000 7feca20a2640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fec880afc60 0x7fec880b2170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec98005fd0 tx=0x7fec98009500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.351+0000 7fec937fe640 1 -- 172.21.15.19:0/1343527663 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec8c009de0 con 0x7fec9c10b650 2024-04-25T10:57:45.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:45 smithi019 ceph-mon[29209]: pgmap v100: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:45.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:45 smithi169 ceph-mon[33049]: pgmap v100: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:45.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:45 smithi112 ceph-mon[32799]: pgmap v100: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.579+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec9c1a72e0 con 0x7fec9c10b650 2024-04-25T10:57:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.579+0000 7fec937fe640 1 -- 172.21.15.19:0/1343527663 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fec8c0a6d70 con 0x7fec9c10b650 2024-04-25T10:57:45.580 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:57:45.580 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:57:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fec880afc60 msgr2=0x7fec880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fec880afc60 0x7fec880b2170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec98005fd0 tx=0x7fec98009500 comp rx=0 tx=0).stop 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10b650 msgr2=0x7fec9c19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10b650 0x7fec9c19aba0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d950 tx=0x7fec8c00de10 comp rx=0 tx=0).stop 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 shutdown_connections 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fec880afc60 0x7fec880b2170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9c10e040 0x7fec9c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec9c10b650 0x7fec9c19aba0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 --2- 172.21.15.19:0/1343527663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec9c104bf0 0x7fec9c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 >> 172.21.15.19:0/1343527663 conn(0x7fec9c078110 msgr2=0x7fec9c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.581+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 shutdown_connections 2024-04-25T10:57:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.582+0000 7feca4306640 1 -- 172.21.15.19:0/1343527663 wait complete. 2024-04-25T10:57:45.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:57:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 -- 172.21.15.19:0/2788544539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370104bf0 msgr2=0x7fb370105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 --2- 172.21.15.19:0/2788544539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370104bf0 0x7fb370105000 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb360009900 tx=0x7fb36002a410 comp rx=0 tx=0).stop 2024-04-25T10:57:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 -- 172.21.15.19:0/2788544539 shutdown_connections 2024-04-25T10:57:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 --2- 172.21.15.19:0/2788544539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb37010e040 0x7fb370110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 --2- 172.21.15.19:0/2788544539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb37010b650 0x7fb37010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 --2- 172.21.15.19:0/2788544539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370104bf0 0x7fb370105000 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 -- 172.21.15.19:0/2788544539 >> 172.21.15.19:0/2788544539 conn(0x7fb370078100 msgr2=0x7fb370078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 -- 172.21.15.19:0/2788544539 shutdown_connections 2024-04-25T10:57:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.707+0000 7fb3765c5640 1 -- 172.21.15.19:0/2788544539 wait complete. 2024-04-25T10:57:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 Processor -- start 2024-04-25T10:57:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 -- start start 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb370104bf0 0x7fb37019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb37010b650 0x7fb37019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb37010e040 0x7fb37019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb370113ca0 con 0x7fb37010b650 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb370113b20 con 0x7fb37010e040 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb3765c5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb370113e20 con 0x7fb370104bf0 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb37010b650 0x7fb37019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb37010b650 0x7fb37019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35008/0 (socket says 172.21.15.19:35008) 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 -- 172.21.15.19:0/671439680 learned_addr learned my addr 172.21.15.19:0/671439680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36ffff640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb370104bf0 0x7fb37019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb374b62640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb37010e040 0x7fb37019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 -- 172.21.15.19:0/671439680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb370104bf0 msgr2=0x7fb37019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb370104bf0 0x7fb37019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 -- 172.21.15.19:0/671439680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb37010e040 msgr2=0x7fb37019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb37010e040 0x7fb37019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36f7fe640 1 -- 172.21.15.19:0/671439680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37019f560 con 0x7fb37010b650 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.708+0000 7fb36ffff640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb370104bf0 0x7fb37019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.709+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb37010b650 0x7fb37019ac60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d950 tx=0x7fb36400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.709+0000 7fb36d7fa640 1 -- 172.21.15.19:0/671439680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364017070 con 0x7fb37010b650 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.709+0000 7fb36d7fa640 1 -- 172.21.15.19:0/671439680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb364009c30 con 0x7fb37010b650 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.709+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37019f850 con 0x7fb37010b650 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.709+0000 7fb36d7fa640 1 -- 172.21.15.19:0/671439680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400f620 con 0x7fb37010b650 2024-04-25T10:57:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.709+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3701a71d0 con 0x7fb37010b650 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.711+0000 7fb37d477740 1 Processor -- start 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.711+0000 7fb36d7fa640 1 -- 172.21.15.19:0/671439680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb364010460 con 0x7fb37010b650 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.711+0000 7fb37d477740 1 -- start start 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb37d477740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4aa390 0x55982d84b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb37d477740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4814d0 0x55982d84bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb37d477740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d8586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb37d477740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55982d7b0d80 con 0x55982d4814d0 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb37d477740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55982d76aa20 con 0x55982d4896b0 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb37d477740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55982d7acd40 con 0x55982d4aa390 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4814d0 0x55982d84bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4814d0 0x55982d84bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35024/0 (socket says 172.21.15.19:35024) 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 -- 172.21.15.19:0/768451586 learned_addr learned my addr 172.21.15.19:0/768451586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d8586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36d7fa640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3580afce0 0x7fb3580b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb374b62640 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4aa390 0x55982d84b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 -- 172.21.15.19:0/768451586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4aa390 msgr2=0x55982d84b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4aa390 0x55982d84b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 -- 172.21.15.19:0/768451586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 msgr2=0x55982d8586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d8586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 -- 172.21.15.19:0/768451586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982d775c30 con 0x55982d4814d0 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb374b62640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3580afce0 0x7fb3580b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36d7fa640 1 -- 172.21.15.19:0/671439680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb3640dec90 con 0x7fb37010b650 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.712+0000 7fb36ffff640 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4814d0 0x55982d84bd50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb36002a930 tx=0x7fb360032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15101c845540630f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.713+0000 7fb3567fc640 1 -- 172.21.15.19:0/768451586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36003f070 con 0x55982d4814d0 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.713+0000 7fb3567fc640 1 -- 172.21.15.19:0/768451586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3600316a0 con 0x55982d4814d0 2024-04-25T10:57:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.713+0000 7fb3567fc640 1 -- 172.21.15.19:0/768451586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360035070 con 0x55982d4814d0 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.713+0000 7fb37d477740 1 -- 172.21.15.19:0/768451586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4814d0 msgr2=0x55982d84bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.713+0000 7fb37d477740 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4814d0 0x55982d84bd50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb36002a930 tx=0x7fb360032c30 comp rx=0 tx=0).stop 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.713+0000 7fb374b62640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3580afce0 0x7fb3580b21f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55982d848980 tx=0x7fb368004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- 172.21.15.19:0/768451586 shutdown_connections 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d8586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4814d0 0x55982d84bd50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 --2- 172.21.15.19:0/768451586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4aa390 0x55982d84b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- 172.21.15.19:0/768451586 >> 172.21.15.19:0/768451586 conn(0x55982d49e4e0 msgr2=0x55982d847fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- 172.21.15.19:0/768451586 shutdown_connections 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- 172.21.15.19:0/768451586 wait complete. 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- start start 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 Processor -- start 2024-04-25T10:57:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4814d0 0x55982d856260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4896b0 0x55982d8567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4aa390 0x55982d8621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55982d858d30 con 0x55982d4aa390 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55982d737dc0 con 0x55982d4814d0 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.714+0000 7fb37d477740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55982d858eb0 con 0x55982d4896b0 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb37d477740 10 client.0 _ll_register_callbacks cb 0x55982d753600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4aa390 0x55982d8621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4aa390 0x55982d8621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35038/0 (socket says 172.21.15.19:35038) 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 -- 172.21.15.19:0/876055424 learned_addr learned my addr 172.21.15.19:0/876055424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb36ffff640 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4814d0 0x55982d856260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4896b0 0x55982d8567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 -- 172.21.15.19:0/876055424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4896b0 msgr2=0x55982d8567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4896b0 0x55982d8567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 -- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4814d0 msgr2=0x55982d856260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4814d0 0x55982d856260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 -- 172.21.15.19:0/876055424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982d4c23f0 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb374b62640 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4aa390 0x55982d8621b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb368011790 tx=0x7fb368004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33dbdf2e80f064a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb34f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb34f7fe640 1 -- 172.21.15.19:0/876055424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368022070 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb34f7fe640 1 -- 172.21.15.19:0/876055424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb368004d20 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3600316a0 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb34f7fe640 1 -- 172.21.15.19:0/876055424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb36802d050 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.715+0000 7fb37d477740 10 client.16008 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55982d58c7d0 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb34f7fe640 1 -- 172.21.15.19:0/876055424 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3600316a0 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb34f7fe640 1 -- 172.21.15.19:0/876055424 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb3680054c0 con 0x55982d4aa390 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb37d477740 10 client.16008 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb37d477740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb37d477740 20 client.16008 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb37d477740 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d468f60 0x55982d8686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb37d477740 4 client.16008 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55982d7be990 con 0x55982d468f60 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.716+0000 7fb36ffff640 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d468f60 0x55982d8686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.717+0000 7fb34f7fe640 10 client.16008 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.717+0000 7fb36ffff640 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d468f60 0x55982d8686e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ced796941ea4ac server_cookie=d9e1121427532768 in_seq=0 out_seq=0 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.717+0000 7fb34f7fe640 1 -- 172.21.15.19:0/876055424 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55982d7be990 con 0x55982d468f60 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.717+0000 7fb34f7fe640 10 client.16008 handle_command_reply: tid=0 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.717+0000 7fb37d477740 1 client.16008 shutdown 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4aa390 msgr2=0x55982d8621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4aa390 0x55982d8621b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb368011790 tx=0x7fb368004ab0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 shutdown_connections 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d468f60 0x55982d8686e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4aa390 0x55982d8621b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4896b0 0x55982d8567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb34f7fe640 0 client.16008 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 --2- 172.21.15.19:0/876055424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4814d0 0x55982d856260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 >> 172.21.15.19:0/876055424 conn(0x55982d49e4e0 msgr2=0x55982d84d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 shutdown_connections 2024-04-25T10:57:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 1 -- 172.21.15.19:0/876055424 wait complete. 2024-04-25T10:57:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.718+0000 7fb37d477740 20 client.16008 trim_cache size 0 max 16384 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 Processor -- start 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 -- start start 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d468f60 0x55982d854f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d84bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4896b0 0x55982d851780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3701ad830 con 0x55982d4896b0 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3701b00b0 con 0x55982d468f60 2024-04-25T10:57:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb37d477740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3701adea0 con 0x55982d4814d0 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4896b0 0x55982d851780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4896b0 0x55982d851780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35046/0 (socket says 172.21.15.19:35046) 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 -- 172.21.15.19:0/2483824952 learned_addr learned my addr 172.21.15.19:0/2483824952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb36ffff640 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d468f60 0x55982d854f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d84bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 -- 172.21.15.19:0/2483824952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 msgr2=0x55982d84bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d84bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 -- 172.21.15.19:0/2483824952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d468f60 msgr2=0x55982d854f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d468f60 0x55982d854f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 -- 172.21.15.19:0/2483824952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982d8434c0 con 0x55982d4896b0 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb36ffff640 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d468f60 0x55982d854f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.732+0000 7fb374b62640 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4896b0 0x55982d851780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb3680114e0 tx=0x7fb36801e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24916027763aaaba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb355ffb640 1 -- 172.21.15.19:0/2483824952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368035070 con 0x55982d4896b0 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb355ffb640 1 -- 172.21.15.19:0/2483824952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36801ee50 con 0x55982d4896b0 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb355ffb640 1 -- 172.21.15.19:0/2483824952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368022070 con 0x55982d4896b0 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 -- 172.21.15.19:0/2483824952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4896b0 msgr2=0x55982d851780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4896b0 0x55982d851780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb3680114e0 tx=0x7fb36801e6b0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 -- 172.21.15.19:0/2483824952 shutdown_connections 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d4896b0 0x55982d851780 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d84bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 --2- 172.21.15.19:0/2483824952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d468f60 0x55982d854f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 -- 172.21.15.19:0/2483824952 >> 172.21.15.19:0/2483824952 conn(0x55982d49e4e0 msgr2=0x55982d84b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 -- 172.21.15.19:0/2483824952 shutdown_connections 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.733+0000 7fb37d477740 1 -- 172.21.15.19:0/2483824952 wait complete. 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 -- start start 2024-04-25T10:57:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 Processor -- start 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d468f60 0x55982d859150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d859690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d859bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3701ae570 con 0x55982d468f60 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3701af9e0 con 0x55982d4896b0 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3701aec40 con 0x55982d4814d0 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb37d477740 10 client.0 _ll_register_callbacks cb 0x7fb368009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d468f60 0x55982d859150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d468f60 0x55982d859150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35056/0 (socket says 172.21.15.19:35056) 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb36ffff640 1 -- 172.21.15.19:0/3641810992 learned_addr learned my addr 172.21.15.19:0/3641810992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.734+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d859690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb374b62640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d859bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb36ffff640 1 -- 172.21.15.19:0/3641810992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 msgr2=0x55982d859690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb36ffff640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d859690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb36ffff640 1 -- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 msgr2=0x55982d859bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb36ffff640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d859bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb36ffff640 1 -- 172.21.15.19:0/3641810992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982d8cec40 con 0x55982d468f60 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb374b62640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d859bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb36ffff640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d468f60 0x55982d859150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb360009900 tx=0x7fb360005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301c9c3c8c3a1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb34f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb34f7fe640 1 -- 172.21.15.19:0/3641810992 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36003f040 con 0x55982d468f60 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb34f7fe640 1 -- 172.21.15.19:0/3641810992 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb360004040 con 0x55982d468f60 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36801ee50 con 0x55982d468f60 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb34f7fe640 1 -- 172.21.15.19:0/3641810992 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb360035040 con 0x55982d468f60 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb37d477740 10 client.16014 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55982d8cff00 con 0x55982d468f60 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.735+0000 7fb34f7fe640 1 -- 172.21.15.19:0/3641810992 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36801ee50 con 0x55982d468f60 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb34f7fe640 1 -- 172.21.15.19:0/3641810992 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb360004240 con 0x55982d468f60 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb37d477740 10 client.16014 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb37d477740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb37d477740 20 client.16014 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb37d477740 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d4aa390 0x55982d8d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb37d477740 4 client.16014 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55982d7b8800 con 0x55982d4aa390 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d4aa390 0x55982d8d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb36f7fe640 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d4aa390 0x55982d8d2250 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1787cfff4cde53b server_cookie=cccd6d2ad241f1ff in_seq=0 out_seq=0 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.736+0000 7fb34f7fe640 10 client.16014 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb34f7fe640 1 -- 172.21.15.19:0/3641810992 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55982d7b8800 con 0x55982d4aa390 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb34f7fe640 10 client.16014 handle_command_reply: tid=0 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 client.16014 shutdown 2024-04-25T10:57:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d468f60 msgr2=0x55982d859150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d468f60 0x55982d859150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb360009900 tx=0x7fb360005d50 comp rx=0 tx=0).stop 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 shutdown_connections 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55982d4aa390 0x55982d8d2250 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982d4896b0 0x55982d859bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55982d4814d0 0x55982d859690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb34f7fe640 0 client.16014 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 --2- 172.21.15.19:0/3641810992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55982d468f60 0x55982d859150 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 >> 172.21.15.19:0/3641810992 conn(0x55982d49e4e0 msgr2=0x55982d8477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 shutdown_connections 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 1 -- 172.21.15.19:0/3641810992 wait complete. 2024-04-25T10:57:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb37d477740 20 client.16014 trim_cache size 0 max 16384 2024-04-25T10:57:45.745 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:57:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3580afce0 msgr2=0x7fb3580b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb3765c5640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3580afce0 0x7fb3580b21f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55982d848980 tx=0x7fb368004a40 comp rx=0 tx=0).stop 2024-04-25T10:57:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb37010b650 msgr2=0x7fb37019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.737+0000 7fb3765c5640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb37010b650 0x7fb37019ac60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d950 tx=0x7fb36400de10 comp rx=0 tx=0).stop 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 shutdown_connections 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3580afce0 0x7fb3580b21f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb37010e040 0x7fb37019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb37010b650 0x7fb37019ac60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 --2- 172.21.15.19:0/671439680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb370104bf0 0x7fb37019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 >> 172.21.15.19:0/671439680 conn(0x7fb370078100 msgr2=0x7fb370101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 shutdown_connections 2024-04-25T10:57:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:45.738+0000 7fb3765c5640 1 -- 172.21.15.19:0/671439680 wait complete. 2024-04-25T10:57:46.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:46 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1343527663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:46.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:46 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1343527663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:46.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:46 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1343527663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:46.750 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:57:46.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:57:46.750 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:57:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 -- 172.21.15.19:0/2218663124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 msgr2=0x7f33ac1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 --2- 172.21.15.19:0/2218663124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac1036b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009900 tx=0x7f33a002a410 comp rx=0 tx=0).stop 2024-04-25T10:57:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 -- 172.21.15.19:0/2218663124 shutdown_connections 2024-04-25T10:57:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 --2- 172.21.15.19:0/2218663124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac104260 0x7f33ac110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 --2- 172.21.15.19:0/2218663124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac109fb0 0x7f33ac103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 --2- 172.21.15.19:0/2218663124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac1036b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 -- 172.21.15.19:0/2218663124 >> 172.21.15.19:0/2218663124 conn(0x7f33ac0ff210 msgr2=0x7f33ac101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 -- 172.21.15.19:0/2218663124 shutdown_connections 2024-04-25T10:57:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.880+0000 7f33b3092640 1 -- 172.21.15.19:0/2218663124 wait complete. 2024-04-25T10:57:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 Processor -- start 2024-04-25T10:57:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 -- start start 2024-04-25T10:57:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac104260 0x7f33ac19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac109fb0 0x7f33ac19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac113d00 con 0x7f33ac109b10 2024-04-25T10:57:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac113b80 con 0x7f33ac109fb0 2024-04-25T10:57:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33b3092640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac113e80 con 0x7f33ac104260 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.881+0000 7f33abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35072/0 (socket says 172.21.15.19:35072) 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33abfff640 1 -- 172.21.15.19:0/1853260597 learned_addr learned my addr 172.21.15.19:0/1853260597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac104260 0x7f33ac19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33b162f640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac109fb0 0x7f33ac19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33abfff640 1 -- 172.21.15.19:0/1853260597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac104260 msgr2=0x7f33ac19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33abfff640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac104260 0x7f33ac19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33abfff640 1 -- 172.21.15.19:0/1853260597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac109fb0 msgr2=0x7f33ac19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33abfff640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac109fb0 0x7f33ac19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33abfff640 1 -- 172.21.15.19:0/1853260597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac19f620 con 0x7f33ac109b10 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33abfff640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac19ad20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3398005d00 tx=0x7f339800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33a9ffb640 1 -- 172.21.15.19:0/1853260597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398017070 con 0x7f33ac109b10 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33a9ffb640 1 -- 172.21.15.19:0/1853260597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3398009c30 con 0x7f33ac109b10 2024-04-25T10:57:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac19f910 con 0x7f33ac109b10 2024-04-25T10:57:46.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.882+0000 7f33a9ffb640 1 -- 172.21.15.19:0/1853260597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800f620 con 0x7f33ac109b10 2024-04-25T10:57:46.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.883+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac19fe10 con 0x7f33ac109b10 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 Processor -- start 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 -- start start 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33a9ffb640 1 -- 172.21.15.19:0/1853260597 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f339801a070 con 0x7f33ac109b10 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e1282620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0eb9550 0x55b6e1282b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e12830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b6e11e8b00 con 0x55b6e0ee2410 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6e1184920 con 0x55b6e0ec1730 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33ba014740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b6e11e4ac0 con 0x55b6e0eb9550 2024-04-25T10:57:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33b162f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e1282620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33b162f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e1282620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35082/0 (socket says 172.21.15.19:35082) 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33b162f640 1 -- 172.21.15.19:0/4124192933 learned_addr learned my addr 172.21.15.19:0/4124192933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.886+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0eb9550 0x55b6e1282b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33abfff640 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e12830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33b162f640 1 -- 172.21.15.19:0/4124192933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0eb9550 msgr2=0x55b6e1282b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33b162f640 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0eb9550 0x55b6e1282b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33b162f640 1 -- 172.21.15.19:0/4124192933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 msgr2=0x55b6e12830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33b162f640 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e12830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33b162f640 1 -- 172.21.15.19:0/4124192933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e0fc27e0 con 0x55b6e0ee2410 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33abfff640 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e12830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33a9ffb640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f339c0afc60 0x7f339c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33b162f640 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e1282620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fc0 tx=0x7f33a4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3b1397b01b8308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f339c0afc60 0x7f339c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f33a9ffb640 1 -- 172.21.15.19:0/1853260597 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f33980dea30 con 0x7f33ac109b10 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f3396ffd640 1 -- 172.21.15.19:0/4124192933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4003b70 con 0x55b6e0ee2410 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f3396ffd640 1 -- 172.21.15.19:0/4124192933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a4003d10 con 0x55b6e0ee2410 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.887+0000 7f3396ffd640 1 -- 172.21.15.19:0/4124192933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4011a40 con 0x55b6e0ee2410 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 -- 172.21.15.19:0/4124192933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 msgr2=0x55b6e1282620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f339c0afc60 0x7f339c0b2170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33a0000c00 tx=0x7f33a0031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e1282620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fc0 tx=0x7f33a4010860 comp rx=0 tx=0).stop 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 -- 172.21.15.19:0/4124192933 shutdown_connections 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e12830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0eb9550 0x55b6e1282b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 --2- 172.21.15.19:0/4124192933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e1282620 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 -- 172.21.15.19:0/4124192933 >> 172.21.15.19:0/4124192933 conn(0x55b6e0ed6560 msgr2=0x55b6e127edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 -- 172.21.15.19:0/4124192933 shutdown_connections 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.888+0000 7f33ba014740 1 -- 172.21.15.19:0/4124192933 wait complete. 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 -- start start 2024-04-25T10:57:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 Processor -- start 2024-04-25T10:57:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e128cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e128d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e1292df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b6e128f970 con 0x55b6e0eb9550 2024-04-25T10:57:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6e11825d0 con 0x55b6e0ec1730 2024-04-25T10:57:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b6e128faf0 con 0x55b6e0ee2410 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33ba014740 10 client.0 _ll_register_callbacks cb 0x55b6e1170080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33b0e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e128cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33b0e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e128cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35084/0 (socket says 172.21.15.19:35084) 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33b0e2e640 1 -- 172.21.15.19:0/1307306230 learned_addr learned my addr 172.21.15.19:0/1307306230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33b162f640 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e1292df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.889+0000 7f33abfff640 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e128d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33b0e2e640 1 -- 172.21.15.19:0/1307306230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 msgr2=0x55b6e1292df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e1292df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33b0e2e640 1 -- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 msgr2=0x55b6e128d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e128d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33b0e2e640 1 -- 172.21.15.19:0/1307306230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e1022f70 con 0x55b6e0eb9550 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e128cc50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009900 tx=0x7f33a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d357aa92f76827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f338bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f338bfff640 1 -- 172.21.15.19:0/1307306230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a004d030 con 0x55b6e0eb9550 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f338bfff640 1 -- 172.21.15.19:0/1307306230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f33a0004250 con 0x55b6e0eb9550 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a4003d10 con 0x55b6e0eb9550 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f338bfff640 1 -- 172.21.15.19:0/1307306230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f33a0035040 con 0x55b6e0eb9550 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33ba014740 10 client.16026 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6e0eba170 con 0x55b6e0eb9550 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.890+0000 7f338bfff640 1 -- 172.21.15.19:0/1307306230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a4003d10 con 0x55b6e0eb9550 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f338bfff640 1 -- 172.21.15.19:0/1307306230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f33a005b5c0 con 0x55b6e0eb9550 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33ba014740 10 client.16026 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33ba014740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33ba014740 20 client.16026 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33ba014740 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0ea11f0 0x55b6e129ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33ba014740 4 client.16026 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6e11f6710 con 0x55b6e0ea11f0 2024-04-25T10:57:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33abfff640 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0ea11f0 0x55b6e129ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f33abfff640 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0ea11f0 0x55b6e129ef60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8843d60e1a51b16 server_cookie=ecab709ca43267a3 in_seq=0 out_seq=0 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.891+0000 7f338bfff640 10 client.16026 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f338bfff640 1 -- 172.21.15.19:0/1307306230 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b6e11f6710 con 0x55b6e0ea11f0 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f338bfff640 10 client.16026 handle_command_reply: tid=0 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 client.16026 shutdown 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 msgr2=0x55b6e128cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e128cc50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009900 tx=0x7f33a0009260 comp rx=0 tx=0).stop 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 shutdown_connections 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0ea11f0 0x55b6e129ef60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e1292df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f338bfff640 0 client.16026 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e128d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 --2- 172.21.15.19:0/1307306230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e128cc50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.892+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 >> 172.21.15.19:0/1307306230 conn(0x55b6e0ed6560 msgr2=0x55b6e1282450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.893+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 shutdown_connections 2024-04-25T10:57:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.893+0000 7f33ba014740 1 -- 172.21.15.19:0/1307306230 wait complete. 2024-04-25T10:57:46.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.893+0000 7f33ba014740 20 client.16026 trim_cache size 0 max 16384 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 Processor -- start 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 -- start start 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e127def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e1292d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e12932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1ad980 con 0x55b6e0eb9550 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b0200 con 0x55b6e0ec1730 2024-04-25T10:57:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.911+0000 7f33ba014740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1adff0 con 0x55b6e0ee2410 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e127def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e127def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35088/0 (socket says 172.21.15.19:35088) 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 -- 172.21.15.19:0/2800007350 learned_addr learned my addr 172.21.15.19:0/2800007350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b162f640 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e12932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33abfff640 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e1292d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 -- 172.21.15.19:0/2800007350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 msgr2=0x55b6e12932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e12932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 -- 172.21.15.19:0/2800007350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 msgr2=0x55b6e1292d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e1292d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 -- 172.21.15.19:0/2800007350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e127ff20 con 0x55b6e0eb9550 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33abfff640 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e1292d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e127def0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f33a00057c0 tx=0x7f33a005be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49216d10be7a49e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33967fc640 1 -- 172.21.15.19:0/2800007350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a004d040 con 0x55b6e0eb9550 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33967fc640 1 -- 172.21.15.19:0/2800007350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a0060070 con 0x55b6e0eb9550 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.912+0000 7f33967fc640 1 -- 172.21.15.19:0/2800007350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a00054f0 con 0x55b6e0eb9550 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 -- 172.21.15.19:0/2800007350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 msgr2=0x55b6e127def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e127def0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f33a00057c0 tx=0x7f33a005be00 comp rx=0 tx=0).stop 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 -- 172.21.15.19:0/2800007350 shutdown_connections 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ee2410 0x55b6e12932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0ec1730 0x55b6e1292d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 --2- 172.21.15.19:0/2800007350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0eb9550 0x55b6e127def0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 -- 172.21.15.19:0/2800007350 >> 172.21.15.19:0/2800007350 conn(0x55b6e0ed6560 msgr2=0x55b6e128b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 -- 172.21.15.19:0/2800007350 shutdown_connections 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 -- 172.21.15.19:0/2800007350 wait complete. 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 -- start start 2024-04-25T10:57:46.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.913+0000 7f33ba014740 1 Processor -- start 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33ba014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0eb9550 0x55b6e12950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33ba014740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ec1730 0x55b6e1295610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33ba014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e13062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33ba014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1ae6c0 con 0x55b6e0ee2410 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33ba014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1afb30 con 0x55b6e0eb9550 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33ba014740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1aed90 con 0x55b6e0ec1730 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33ba014740 10 client.0 _ll_register_callbacks cb 0x7f33a0005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:46.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e13062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e13062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35104/0 (socket says 172.21.15.19:35104) 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 -- 172.21.15.19:0/4002690768 learned_addr learned my addr 172.21.15.19:0/4002690768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0eb9550 0x55b6e12950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 -- 172.21.15.19:0/4002690768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ec1730 msgr2=0x55b6e1295610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33abfff640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ec1730 0x55b6e1295610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ec1730 0x55b6e1295610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 -- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0eb9550 msgr2=0x55b6e12950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0eb9550 0x55b6e12950d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b162f640 1 -- 172.21.15.19:0/4002690768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e1306810 con 0x55b6e0ee2410 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.914+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0eb9550 0x55b6e12950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33b162f640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e13062d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011610 tx=0x7f33a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761dfe477dcbd892 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f338bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f338bfff640 1 -- 172.21.15.19:0/4002690768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4011e10 con 0x55b6e0ee2410 2024-04-25T10:57:46.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f338bfff640 1 -- 172.21.15.19:0/4002690768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f33a4005040 con 0x55b6e0ee2410 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a0060070 con 0x55b6e0ee2410 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f338bfff640 1 -- 172.21.15.19:0/4002690768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f33a4035050 con 0x55b6e0ee2410 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 10 client.16032 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6e13079a0 con 0x55b6e0ee2410 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f338bfff640 1 -- 172.21.15.19:0/4002690768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a0060070 con 0x55b6e0ee2410 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f338bfff640 1 -- 172.21.15.19:0/4002690768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f33a401b440 con 0x55b6e0ee2410 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 10 client.16032 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 20 client.16032 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0e76630 0x55b6e1309ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 4 client.16032 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.915+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b6e11f0580 con 0x55b6e0e76630 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.916+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0e76630 0x55b6e1309ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.916+0000 7f338bfff640 10 client.16032 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.916+0000 7f33b0e2e640 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0e76630 0x55b6e1309ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12196506ba0a1d3e server_cookie=3a1827f3276fb16 in_seq=0 out_seq=0 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f338bfff640 1 -- 172.21.15.19:0/4002690768 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b6e11f0580 con 0x55b6e0e76630 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f338bfff640 10 client.16032 handle_command_reply: tid=0 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 client.16032 shutdown 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 msgr2=0x55b6e13062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e13062d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011610 tx=0x7f33a40115a0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 shutdown_connections 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b6e0e76630 0x55b6e1309ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b6e0ee2410 0x55b6e13062d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f338bfff640 0 client.16032 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e0ec1730 0x55b6e1295610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 --2- 172.21.15.19:0/4002690768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6e0eb9550 0x55b6e12950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 >> 172.21.15.19:0/4002690768 conn(0x55b6e0ed6560 msgr2=0x55b6e127dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 shutdown_connections 2024-04-25T10:57:46.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 1 -- 172.21.15.19:0/4002690768 wait complete. 2024-04-25T10:57:46.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.918+0000 7f33ba014740 20 client.16032 trim_cache size 0 max 16384 2024-04-25T10:57:46.924 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:46.924 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:57:46.924 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "cd7af6f8-d85f-4ddc-9a48-3319b3845e64", 2024-04-25T10:57:46.924 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:57:46.924 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f339c0afc60 msgr2=0x7f339c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f339c0afc60 0x7f339c0b2170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33a0000c00 tx=0x7f33a0031400 comp rx=0 tx=0).stop 2024-04-25T10:57:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 msgr2=0x7f33ac19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac19ad20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3398005d00 tx=0x7f339800dde0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 shutdown_connections 2024-04-25T10:57:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f339c0afc60 0x7f339c0b2170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac109fb0 0x7f33ac19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac109b10 0x7f33ac19ad20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 --2- 172.21.15.19:0/1853260597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac104260 0x7f33ac19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 >> 172.21.15.19:0/1853260597 conn(0x7f33ac0ff210 msgr2=0x7f33ac0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 shutdown_connections 2024-04-25T10:57:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:46.919+0000 7f33b3092640 1 -- 172.21.15.19:0/1853260597 wait complete. 2024-04-25T10:57:46.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:57:46.931 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:57:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 -- 172.21.15.19:0/2188790047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf01031c0 msgr2=0x7facf010f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 --2- 172.21.15.19:0/2188790047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf01031c0 0x7facf010f4a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7face8009ca0 tx=0x7face802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 -- 172.21.15.19:0/2188790047 shutdown_connections 2024-04-25T10:57:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 --2- 172.21.15.19:0/2188790047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf01031c0 0x7facf010f4a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 --2- 172.21.15.19:0/2188790047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf0108eb0 0x7facf0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 --2- 172.21.15.19:0/2188790047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0108a10 0x7facf0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 -- 172.21.15.19:0/2188790047 >> 172.21.15.19:0/2188790047 conn(0x7facf0078110 msgr2=0x7facf0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 -- 172.21.15.19:0/2188790047 shutdown_connections 2024-04-25T10:57:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.044+0000 7facf5772640 1 -- 172.21.15.19:0/2188790047 wait complete. 2024-04-25T10:57:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 Processor -- start 2024-04-25T10:57:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 -- start start 2024-04-25T10:57:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf01031c0 0x7facf0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0108a10 0x7facf0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf0108eb0 0x7facf019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112c50 con 0x7facf0108eb0 2024-04-25T10:57:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112ad0 con 0x7facf0108a10 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facf5772640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112dd0 con 0x7facf01031c0 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf0108eb0 0x7facf019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf0108eb0 0x7facf019de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35122/0 (socket says 172.21.15.19:35122) 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facef7fe640 1 -- 172.21.15.19:0/36808547 learned_addr learned my addr 172.21.15.19:0/36808547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7facee7fc640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0108a10 0x7facf0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.045+0000 7faceeffd640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf01031c0 0x7facf0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facef7fe640 1 -- 172.21.15.19:0/36808547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf01031c0 msgr2=0x7facf0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facef7fe640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf01031c0 0x7facf0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facef7fe640 1 -- 172.21.15.19:0/36808547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0108a10 msgr2=0x7facf0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facef7fe640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0108a10 0x7facf0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facef7fe640 1 -- 172.21.15.19:0/36808547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf019e490 con 0x7facf0108eb0 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7faceeffd640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf01031c0 0x7facf0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facef7fe640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf0108eb0 0x7facf019de50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7face8005cc0 tx=0x7face80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facd7fff640 1 -- 172.21.15.19:0/36808547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face804a070 con 0x7facf0108eb0 2024-04-25T10:57:47.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf019e720 con 0x7facf0108eb0 2024-04-25T10:57:47.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facd7fff640 1 -- 172.21.15.19:0/36808547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face8003b80 con 0x7facf0108eb0 2024-04-25T10:57:47.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facd7fff640 1 -- 172.21.15.19:0/36808547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face802f990 con 0x7facf0108eb0 2024-04-25T10:57:47.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.046+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf019ec70 con 0x7facf0108eb0 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 Processor -- start 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facd7fff640 1 -- 172.21.15.19:0/36808547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7face804d070 con 0x7facf0108eb0 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 -- start start 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 0x564c98794a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c98794fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564c986fad80 con 0x564c983d36b0 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c986b48b0 con 0x564c983cb4d0 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.049+0000 7facfc627740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564c986f6d40 con 0x564c983f4390 2024-04-25T10:57:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35132/0 (socket says 172.21.15.19:35132) 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 -- 172.21.15.19:0/4155198634 learned_addr learned my addr 172.21.15.19:0/4155198634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7faceeffd640 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c98794fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 0x564c98794a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facd7fff640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7facd80afc60 0x7facd80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 -- 172.21.15.19:0/4155198634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 msgr2=0x564c98794a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 0x564c98794a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 -- 172.21.15.19:0/4155198634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 msgr2=0x564c98794fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c98794fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 -- 172.21.15.19:0/4155198634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c98685a40 con 0x564c983d36b0 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7faceeffd640 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c98794fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:57:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facef7fe640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7facd80afc60 0x7facd80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facd7fff640 1 -- 172.21.15.19:0/36808547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7face81016e0 con 0x7facf0108eb0 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facee7fc640 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a18f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7face4005d00 tx=0x7face4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dae59d9cc9ba0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.050+0000 7facd57fa640 1 -- 172.21.15.19:0/4155198634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4005220 con 0x564c983d36b0 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facd57fa640 1 -- 172.21.15.19:0/4155198634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face4004c50 con 0x564c983d36b0 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facd57fa640 1 -- 172.21.15.19:0/4155198634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face401d900 con 0x564c983d36b0 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facef7fe640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7facd80afc60 0x7facd80b2170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7face80ff020 tx=0x7face8097220 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 -- 172.21.15.19:0/4155198634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 msgr2=0x564c987a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a18f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7face4005d00 tx=0x7face4005c90 comp rx=0 tx=0).stop 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 -- 172.21.15.19:0/4155198634 shutdown_connections 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a18f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c98794fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 --2- 172.21.15.19:0/4155198634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 0x564c98794a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 -- 172.21.15.19:0/4155198634 >> 172.21.15.19:0/4155198634 conn(0x564c983e84e0 msgr2=0x564c98791210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.051+0000 7facfc627740 1 -- 172.21.15.19:0/4155198634 shutdown_connections 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 -- 172.21.15.19:0/4155198634 wait complete. 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 -- start start 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 Processor -- start 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c987a1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c9879f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564c987a1fe0 con 0x564c983d36b0 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c98681d10 con 0x564c983f4390 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564c987a2160 con 0x564c983cb4d0 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facfc627740 10 client.0 _ll_register_callbacks cb 0x564c986ae210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7faceeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7faceeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35144/0 (socket says 172.21.15.19:35144) 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7faceeffd640 1 -- 172.21.15.19:0/631042625 learned_addr learned my addr 172.21.15.19:0/631042625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.052+0000 7facee7fc640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c987a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facee7fc640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c9879f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7faceeffd640 1 -- 172.21.15.19:0/631042625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 msgr2=0x564c987a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7faceeffd640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c987a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7faceeffd640 1 -- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 msgr2=0x564c9879f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7faceeffd640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c9879f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7faceeffd640 1 -- 172.21.15.19:0/631042625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c984d67d0 con 0x564c983d36b0 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facee7fc640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c9879f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7faceeffd640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a1560 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7face000bf80 tx=0x7face00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd44d7f5706bdccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facd2ff5640 1 -- 172.21.15.19:0/631042625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0004230 con 0x564c983d36b0 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facd2ff5640 1 -- 172.21.15.19:0/631042625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7face00043d0 con 0x564c983d36b0 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face4004c50 con 0x564c983d36b0 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facd2ff5640 1 -- 172.21.15.19:0/631042625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7face00107c0 con 0x564c983d36b0 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facfc627740 10 client.16044 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564c9840c3f0 con 0x564c983d36b0 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facd2ff5640 1 -- 172.21.15.19:0/631042625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face4004c50 con 0x564c983d36b0 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.053+0000 7facd2ff5640 1 -- 172.21.15.19:0/631042625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7face0019070 con 0x564c983d36b0 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facfc627740 10 client.16044 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facfc627740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facfc627740 20 client.16044 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facfc627740 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983b2f60 0x564c987b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facfc627740 4 client.16044 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c98708990 con 0x564c983b2f60 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facee7fc640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983b2f60 0x564c987b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facd2ff5640 10 client.16044 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.054+0000 7facee7fc640 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983b2f60 0x564c987b18c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50eea251820e489 server_cookie=73781fbf58f9ea8b in_seq=0 out_seq=0 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facd2ff5640 1 -- 172.21.15.19:0/631042625 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c98708990 con 0x564c983b2f60 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facd2ff5640 10 client.16044 handle_command_reply: tid=0 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 client.16044 shutdown 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 msgr2=0x564c987a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a1560 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7face000bf80 tx=0x7face00128e0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 shutdown_connections 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983b2f60 0x564c987b18c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c9879f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a1560 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facd2ff5640 0 client.16044 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 --2- 172.21.15.19:0/631042625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c987a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.055+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 >> 172.21.15.19:0/631042625 conn(0x564c983e84e0 msgr2=0x564c98796780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.056+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 shutdown_connections 2024-04-25T10:57:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.056+0000 7facfc627740 1 -- 172.21.15.19:0/631042625 wait complete. 2024-04-25T10:57:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.056+0000 7facfc627740 20 client.16044 trim_cache size 0 max 16384 2024-04-25T10:57:47.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 Processor -- start 2024-04-25T10:57:47.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 -- start start 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c98791280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c98794f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c987b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ab700 con 0x564c983d36b0 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facf01adf80 con 0x564c983f4390 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7facfc627740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facf01abd70 con 0x564c983cb4d0 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7faceeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c98794f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7faceeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c98794f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35146/0 (socket says 172.21.15.19:35146) 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.074+0000 7faceeffd640 1 -- 172.21.15.19:0/3538719542 learned_addr learned my addr 172.21.15.19:0/3538719542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7faceeffd640 1 -- 172.21.15.19:0/3538719542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 msgr2=0x564c98791280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7facee7fc640 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c98791280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7faceeffd640 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c98791280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7facee7fc640 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c987b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7faceeffd640 1 -- 172.21.15.19:0/3538719542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 msgr2=0x564c987b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7faceeffd640 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c987b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7faceeffd640 1 -- 172.21.15.19:0/3538719542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c983cc0f0 con 0x564c983d36b0 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7faceeffd640 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c98794f00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7face000bc20 tx=0x7face000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fd63e3f35d9385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7facd4ff9640 1 -- 172.21.15.19:0/3538719542 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0031070 con 0x564c983d36b0 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7facd4ff9640 1 -- 172.21.15.19:0/3538719542 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face00035f0 con 0x564c983d36b0 2024-04-25T10:57:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.075+0000 7facd4ff9640 1 -- 172.21.15.19:0/3538719542 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0028070 con 0x564c983d36b0 2024-04-25T10:57:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 -- 172.21.15.19:0/3538719542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 msgr2=0x564c98794f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c98794f00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7face000bc20 tx=0x7face000bc50 comp rx=0 tx=0).stop 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 -- 172.21.15.19:0/3538719542 shutdown_connections 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983f4390 0x564c987b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c98794f00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 --2- 172.21.15.19:0/3538719542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983cb4d0 0x564c98791280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 -- 172.21.15.19:0/3538719542 >> 172.21.15.19:0/3538719542 conn(0x564c983e84e0 msgr2=0x564c987947f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 -- 172.21.15.19:0/3538719542 shutdown_connections 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 -- 172.21.15.19:0/3538719542 wait complete. 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 -- start start 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.076+0000 7facfc627740 1 Processor -- start 2024-04-25T10:57:47.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c987a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 0x564c98817c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ac440 con 0x564c983d36b0 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ad8b0 con 0x564c983cb4d0 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facf01acb10 con 0x564c983f4390 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 10 client.0 _ll_register_callbacks cb 0x7face000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35162/0 (socket says 172.21.15.19:35162) 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 -- 172.21.15.19:0/3057918910 learned_addr learned my addr 172.21.15.19:0/3057918910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facee7fc640 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c987a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 -- 172.21.15.19:0/3057918910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 msgr2=0x564c98817c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 0x564c98817c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 -- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 msgr2=0x564c987a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c987a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 -- 172.21.15.19:0/3057918910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c986bb4b0 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facee7fc640 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c987a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7faceeffd640 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a4b80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7face0010b40 tx=0x7face000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0b7bb91ba42c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facd2ff5640 1 -- 172.21.15.19:0/3057918910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0028070 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facd2ff5640 1 -- 172.21.15.19:0/3057918910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7face0003f10 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face00035f0 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facd2ff5640 1 -- 172.21.15.19:0/3057918910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7face0031040 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.077+0000 7facfc627740 10 client.16050 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564c98819070 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facd2ff5640 1 -- 172.21.15.19:0/3057918910 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face00035f0 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facd2ff5640 1 -- 172.21.15.19:0/3057918910 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7face0034070 con 0x564c983d36b0 2024-04-25T10:57:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facfc627740 10 client.16050 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facfc627740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facfc627740 20 client.16050 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facfc627740 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983883a0 0x564c9881b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facfc627740 4 client.16050 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c98702800 con 0x564c983883a0 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.078+0000 7facee7fc640 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983883a0 0x564c9881b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facd2ff5640 10 client.16050 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facee7fc640 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983883a0 0x564c9881b7c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d318e2eac2662a server_cookie=e1170ce7dc47ea70 in_seq=0 out_seq=0 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facd2ff5640 1 -- 172.21.15.19:0/3057918910 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564c98702800 con 0x564c983883a0 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facd2ff5640 10 client.16050 handle_command_reply: tid=0 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 client.16050 shutdown 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 msgr2=0x564c987a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a4b80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7face0010b40 tx=0x7face000b6c0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 shutdown_connections 2024-04-25T10:57:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x564c983883a0 0x564c9881b7c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564c983f4390 0x564c98817c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facd2ff5640 0 client.16050 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c983d36b0 0x564c987a4b80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 --2- 172.21.15.19:0/3057918910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c983cb4d0 0x564c987a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 >> 172.21.15.19:0/3057918910 conn(0x564c983e84e0 msgr2=0x564c9879ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 shutdown_connections 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 1 -- 172.21.15.19:0/3057918910 wait complete. 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.079+0000 7facfc627740 20 client.16050 trim_cache size 0 max 16384 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:47.086 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:57:47.087 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:57:47.087 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7facd80afc60 msgr2=0x7facd80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7facd80afc60 0x7facd80b2170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7face80ff020 tx=0x7face8097220 comp rx=0 tx=0).stop 2024-04-25T10:57:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf0108eb0 msgr2=0x7facf019de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf0108eb0 0x7facf019de50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7face8005cc0 tx=0x7face80319f0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 shutdown_connections 2024-04-25T10:57:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7facd80afc60 0x7facd80b2170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facf0108eb0 0x7facf019de50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0108a10 0x7facf0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 --2- 172.21.15.19:0/36808547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf01031c0 0x7facf0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 >> 172.21.15.19:0/36808547 conn(0x7facf0078110 msgr2=0x7facf0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 shutdown_connections 2024-04-25T10:57:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.080+0000 7facf5772640 1 -- 172.21.15.19:0/36808547 wait complete. 2024-04-25T10:57:47.092 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:57:47.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 -- 172.21.15.19:0/381612344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109fd0 msgr2=0x7f3558103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 --2- 172.21.15.19:0/381612344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109fd0 0x7f3558103c10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3540009900 tx=0x7f354002a410 comp rx=0 tx=0).stop 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 -- 172.21.15.19:0/381612344 shutdown_connections 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 --2- 172.21.15.19:0/381612344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3558104280 0x7f3558110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 --2- 172.21.15.19:0/381612344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109fd0 0x7f3558103c10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 --2- 172.21.15.19:0/381612344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3558109b30 0x7f35581036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 -- 172.21.15.19:0/381612344 >> 172.21.15.19:0/381612344 conn(0x7f35580ff210 msgr2=0x7f3558101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 -- 172.21.15.19:0/381612344 shutdown_connections 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.204+0000 7f355c9b7640 1 -- 172.21.15.19:0/381612344 wait complete. 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 Processor -- start 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 -- start start 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3558104280 0x7f355819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109b30 0x7f355819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3558109fd0 0x7f355819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3558113d20 con 0x7f3558109b30 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3558113ba0 con 0x7f3558104280 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355c9b7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3558113ea0 con 0x7f3558109fd0 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109b30 0x7f355819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109b30 0x7f355819ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35186/0 (socket says 172.21.15.19:35186) 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 -- 172.21.15.19:0/3405197470 learned_addr learned my addr 172.21.15.19:0/3405197470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f355659c640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3558104280 0x7f355819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 -- 172.21.15.19:0/3405197470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3558109fd0 msgr2=0x7f355819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3556d9d640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3558109fd0 0x7f355819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3558109fd0 0x7f355819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 -- 172.21.15.19:0/3405197470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3558104280 msgr2=0x7f355819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3558104280 0x7f355819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3555d9b640 1 -- 172.21.15.19:0/3405197470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355819f5e0 con 0x7f3558109b30 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.205+0000 7f3556d9d640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3558109fd0 0x7f355819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.206+0000 7f3555d9b640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109b30 0x7f355819ace0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f354002a930 tx=0x7f3540004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.206+0000 7f354f7fe640 1 -- 172.21.15.19:0/3405197470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354003f030 con 0x7f3558109b30 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.206+0000 7f354f7fe640 1 -- 172.21.15.19:0/3405197470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3540003aa0 con 0x7f3558109b30 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.206+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355819f870 con 0x7f3558109b30 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.206+0000 7f354f7fe640 1 -- 172.21.15.19:0/3405197470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540035070 con 0x7f3558109b30 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.206+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355819fd60 con 0x7f3558109b30 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.207+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355806a190 con 0x7f3558109b30 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.208+0000 7f354f7fe640 1 -- 172.21.15.19:0/3405197470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3540042030 con 0x7f3558109b30 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.209+0000 7f354f7fe640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f35440afc60 0x7f35440b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.209+0000 7f355659c640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f35440afc60 0x7f35440b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.209+0000 7f354f7fe640 1 -- 172.21.15.19:0/3405197470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f35400ffb40 con 0x7f3558109b30 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.210+0000 7f355659c640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f35440afc60 0x7f35440b2170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3558104810 tx=0x7f3548009680 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.213+0000 7f354f7fe640 1 -- 172.21.15.19:0/3405197470 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35400c8410 con 0x7f3558109b30 2024-04-25T10:57:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:47 smithi019 ceph-mon[29209]: pgmap v101: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:47 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:57:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:47 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:47 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:57:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:47 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:47.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:47 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:57:47.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.439+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35581a7310 con 0x7f3558109b30 2024-04-25T10:57:47.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.439+0000 7f354f7fe640 1 -- 172.21.15.19:0/3405197470 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f35400c7d10 con 0x7f3558109b30 2024-04-25T10:57:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:57:47.440 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:57:47.440 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:57:47.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f35440afc60 msgr2=0x7f35440b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f35440afc60 0x7f35440b2170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3558104810 tx=0x7f3548009680 comp rx=0 tx=0).stop 2024-04-25T10:57:47.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109b30 msgr2=0x7f355819ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109b30 0x7f355819ace0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f354002a930 tx=0x7f3540004270 comp rx=0 tx=0).stop 2024-04-25T10:57:47.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 shutdown_connections 2024-04-25T10:57:47.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f35440afc60 0x7f35440b2170 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3558109fd0 0x7f355819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3558109b30 0x7f355819ace0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 --2- 172.21.15.19:0/3405197470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3558104280 0x7f355819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 >> 172.21.15.19:0/3405197470 conn(0x7f35580ff210 msgr2=0x7f355810f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 shutdown_connections 2024-04-25T10:57:47.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.442+0000 7f355c9b7640 1 -- 172.21.15.19:0/3405197470 wait complete. 2024-04-25T10:57:47.454 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:57:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:47 smithi169 ceph-mon[33049]: pgmap v101: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:47 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:57:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:47 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:47 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:57:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:47 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:47 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:57:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:47 smithi112 ceph-mon[32799]: pgmap v101: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:57:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:47 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:57:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:47 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:47 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:57:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:47 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:47.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:47 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:57:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.567+0000 7f412bff8640 1 -- 172.21.15.19:0/1169438736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124104bf0 msgr2=0x7f4124105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.567+0000 7f412bff8640 1 --2- 172.21.15.19:0/1169438736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124104bf0 0x7f4124105000 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4120009ca0 tx=0x7f412002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 -- 172.21.15.19:0/1169438736 shutdown_connections 2024-04-25T10:57:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 --2- 172.21.15.19:0/1169438736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f412410e040 0x7f4124110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 --2- 172.21.15.19:0/1169438736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f412410b650 0x7f412410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 --2- 172.21.15.19:0/1169438736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124104bf0 0x7f4124105000 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 -- 172.21.15.19:0/1169438736 >> 172.21.15.19:0/1169438736 conn(0x7f4124078100 msgr2=0x7f4124078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 -- 172.21.15.19:0/1169438736 shutdown_connections 2024-04-25T10:57:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 -- 172.21.15.19:0/1169438736 wait complete. 2024-04-25T10:57:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 Processor -- start 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.568+0000 7f412bff8640 1 -- start start 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124104bf0 0x7f412419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f412410b650 0x7f412419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f412410e040 0x7f412419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4124113ce0 con 0x7f412410b650 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4124113b60 con 0x7f4124104bf0 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4124113e60 con 0x7f412410e040 2024-04-25T10:57:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f412410b650 0x7f412419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f412410b650 0x7f412419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35202/0 (socket says 172.21.15.19:35202) 2024-04-25T10:57:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 -- 172.21.15.19:0/2463221901 learned_addr learned my addr 172.21.15.19:0/2463221901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129d94640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124104bf0 0x7f412419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 -- 172.21.15.19:0/2463221901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f412410e040 msgr2=0x7f412419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f412410e040 0x7f412419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 -- 172.21.15.19:0/2463221901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124104bf0 msgr2=0x7f412419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124104bf0 0x7f412419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 -- 172.21.15.19:0/2463221901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412419f5a0 con 0x7f412410b650 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129d94640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124104bf0 0x7f412419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f4129593640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f412410b650 0x7f412419aca0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f411400c890 tx=0x7f411400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f411affd640 1 -- 172.21.15.19:0/2463221901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114004220 con 0x7f412410b650 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f411affd640 1 -- 172.21.15.19:0/2463221901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41140043c0 con 0x7f412410b650 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412419f890 con 0x7f412410b650 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f411affd640 1 -- 172.21.15.19:0/2463221901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114004a00 con 0x7f412410b650 2024-04-25T10:57:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.569+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a7210 con 0x7f412410b650 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 Processor -- start 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 -- start start 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f411affd640 1 -- 172.21.15.19:0/2463221901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f41140079b0 con 0x7f412410b650 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d53795810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533cb4d0 0x559d53795d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d536fad80 con 0x559d533cb4d0 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d536b4a20 con 0x559d533d36b0 2024-04-25T10:57:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4132f89740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559d536f6d40 con 0x559d533f4390 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4129d94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533cb4d0 0x559d53795d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.572+0000 7f4129d94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533cb4d0 0x559d53795d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35216/0 (socket says 172.21.15.19:35216) 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129d94640 1 -- 172.21.15.19:0/4100889219 learned_addr learned my addr 172.21.15.19:0/4100889219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129593640 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f412a595640 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d53795810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129d94640 1 -- 172.21.15.19:0/4100889219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 msgr2=0x559d53795810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129d94640 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d53795810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129d94640 1 -- 172.21.15.19:0/4100889219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 msgr2=0x559d537a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129d94640 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129d94640 1 -- 172.21.15.19:0/4100889219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d536bfc30 con 0x559d533cb4d0 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f411affd640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41100afce0 0x7f41100b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f4129d94640 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533cb4d0 0x559d53795d50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4120009a20 tx=0x7f4120031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fb55db83995cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f412a595640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41100afce0 0x7f41100b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f411affd640 1 -- 172.21.15.19:0/2463221901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f41140dedb0 con 0x7f412410b650 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f410ffff640 1 -- 172.21.15.19:0/4100889219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120031450 con 0x559d533cb4d0 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f410ffff640 1 -- 172.21.15.19:0/4100889219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4120036070 con 0x559d533cb4d0 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.573+0000 7f410ffff640 1 -- 172.21.15.19:0/4100889219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200043a0 con 0x559d533cb4d0 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 -- 172.21.15.19:0/4100889219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533cb4d0 msgr2=0x559d53795d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f412a595640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41100afce0 0x7f41100b21f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x559d53792980 tx=0x7f411c006f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533cb4d0 0x559d53795d50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4120009a20 tx=0x7f4120031af0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 -- 172.21.15.19:0/4100889219 shutdown_connections 2024-04-25T10:57:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533cb4d0 0x559d53795d50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 --2- 172.21.15.19:0/4100889219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d53795810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 -- 172.21.15.19:0/4100889219 >> 172.21.15.19:0/4100889219 conn(0x559d533e84e0 msgr2=0x559d53791fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.574+0000 7f4132f89740 1 -- 172.21.15.19:0/4100889219 shutdown_connections 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 -- 172.21.15.19:0/4100889219 wait complete. 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 -- start start 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 Processor -- start 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533cb4d0 0x559d537a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533f4390 0x559d537ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d537a2d30 con 0x559d533f4390 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d53681dc0 con 0x559d533d36b0 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559d537a2eb0 con 0x559d533cb4d0 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.575+0000 7f4132f89740 10 client.0 _ll_register_callbacks cb 0x559d5369d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533f4390 0x559d537ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533f4390 0x559d537ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35224/0 (socket says 172.21.15.19:35224) 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 -- 172.21.15.19:0/3406364105 learned_addr learned my addr 172.21.15.19:0/3406364105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f4129d94640 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533cb4d0 0x559d537a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f4129593640 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 -- 172.21.15.19:0/3406364105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533cb4d0 msgr2=0x559d537a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533cb4d0 0x559d537a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 -- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 msgr2=0x559d537a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 -- 172.21.15.19:0/3406364105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5340c3f0 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f412a595640 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533f4390 0x559d537ac1b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f411c015730 tx=0x7f411c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685cfd37c0764bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f410d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f410d7fa640 1 -- 172.21.15.19:0/3406364105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c013720 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f410d7fa640 1 -- 172.21.15.19:0/3406364105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f411c0138c0 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4120036070 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.576+0000 7f410d7fa640 1 -- 172.21.15.19:0/3406364105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f411c013ac0 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 10 client.16068 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559d534d67d0 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f410d7fa640 1 -- 172.21.15.19:0/3406364105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4120036070 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f410d7fa640 1 -- 172.21.15.19:0/3406364105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f411c0143e0 con 0x559d533f4390 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 10 client.16068 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 20 client.16068 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533b2f60 0x559d537b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 4 client.16068 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d53708990 con 0x559d533b2f60 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.577+0000 7f4129d94640 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533b2f60 0x559d537b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.578+0000 7f410d7fa640 10 client.16068 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.578+0000 7f4129d94640 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533b2f60 0x559d537b2640 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb588e30f85efa86 server_cookie=90b8797cbb80337a in_seq=0 out_seq=0 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.578+0000 7f410d7fa640 1 -- 172.21.15.19:0/3406364105 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d53708990 con 0x559d533b2f60 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.578+0000 7f410d7fa640 10 client.16068 handle_command_reply: tid=0 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.578+0000 7f4132f89740 1 client.16068 shutdown 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533f4390 msgr2=0x559d537ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533f4390 0x559d537ac1b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f411c015730 tx=0x7f411c004840 comp rx=0 tx=0).stop 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 shutdown_connections 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533b2f60 0x559d537b2640 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533f4390 0x559d537ac1b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533d36b0 0x559d537a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 --2- 172.21.15.19:0/3406364105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533cb4d0 0x559d537a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f410d7fa640 0 client.16068 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 >> 172.21.15.19:0/3406364105 conn(0x559d533e84e0 msgr2=0x559d53797530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 shutdown_connections 2024-04-25T10:57:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 1 -- 172.21.15.19:0/3406364105 wait complete. 2024-04-25T10:57:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.579+0000 7f4132f89740 20 client.16068 trim_cache size 0 max 16384 2024-04-25T10:57:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.592+0000 7f4132f89740 1 Processor -- start 2024-04-25T10:57:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.592+0000 7f4132f89740 1 -- start start 2024-04-25T10:57:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4132f89740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d5379f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4132f89740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4132f89740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d5379a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4132f89740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ad9f0 con 0x559d533d36b0 2024-04-25T10:57:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4132f89740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41241b0270 con 0x559d533cb4d0 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4132f89740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ae060 con 0x559d533f4390 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35230/0 (socket says 172.21.15.19:35230) 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 -- 172.21.15.19:0/1823896015 learned_addr learned my addr 172.21.15.19:0/1823896015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129d94640 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d5379f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f412a595640 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d5379a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 -- 172.21.15.19:0/1823896015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 msgr2=0x559d5379a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d5379a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 -- 172.21.15.19:0/1823896015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 msgr2=0x559d5379f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d5379f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 -- 172.21.15.19:0/1823896015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d537b29d0 con 0x559d533d36b0 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f4129593640 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537abe10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f41140a3000 tx=0x7f4114005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebacb13d7ea408b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.593+0000 7f410f7fe640 1 -- 172.21.15.19:0/1823896015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411401d070 con 0x559d533d36b0 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f410f7fe640 1 -- 172.21.15.19:0/1823896015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4114005290 con 0x559d533d36b0 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f410f7fe640 1 -- 172.21.15.19:0/1823896015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140ab330 con 0x559d533d36b0 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 -- 172.21.15.19:0/1823896015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 msgr2=0x559d537abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537abe10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f41140a3000 tx=0x7f4114005760 comp rx=0 tx=0).stop 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 -- 172.21.15.19:0/1823896015 shutdown_connections 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d5379a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537abe10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 --2- 172.21.15.19:0/1823896015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d5379f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 -- 172.21.15.19:0/1823896015 >> 172.21.15.19:0/1823896015 conn(0x559d533e84e0 msgr2=0x559d5379e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 -- 172.21.15.19:0/1823896015 shutdown_connections 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.594+0000 7f4132f89740 1 -- 172.21.15.19:0/1823896015 wait complete. 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 -- start start 2024-04-25T10:57:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 Processor -- start 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d537a8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d537a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ae730 con 0x559d533d36b0 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41241afba0 con 0x559d533cb4d0 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f41241aee00 con 0x559d533f4390 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4132f89740 10 client.0 _ll_register_callbacks cb 0x7f411c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4129593640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4129593640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35232/0 (socket says 172.21.15.19:35232) 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f4129593640 1 -- 172.21.15.19:0/1091477763 learned_addr learned my addr 172.21.15.19:0/1091477763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.595+0000 7f412a595640 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d537a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4129d94640 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d537a8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4129593640 1 -- 172.21.15.19:0/1091477763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 msgr2=0x559d537a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4129593640 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d537a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4129593640 1 -- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 msgr2=0x559d537a8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4129593640 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d537a8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4129593640 1 -- 172.21.15.19:0/1091477763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d537a4790 con 0x559d533d36b0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4129593640 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537a3ac0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f41140dc5e0 tx=0x7f41140076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c1efc7fbc3f804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f410d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f410d7fa640 1 -- 172.21.15.19:0/1091477763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140c6060 con 0x559d533d36b0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f410d7fa640 1 -- 172.21.15.19:0/1091477763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f41140e2b90 con 0x559d533d36b0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d537960e0 con 0x559d533d36b0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f410d7fa640 1 -- 172.21.15.19:0/1091477763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41140c1030 con 0x559d533d36b0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4132f89740 10 client.16074 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559d5381a950 con 0x559d533d36b0 2024-04-25T10:57:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.596+0000 7f410d7fa640 1 -- 172.21.15.19:0/1091477763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559d537960e0 con 0x559d533d36b0 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f410d7fa640 1 -- 172.21.15.19:0/1091477763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f41140a28f0 con 0x559d533d36b0 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4132f89740 10 client.16074 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4132f89740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4132f89740 20 client.16074 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4132f89740 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533883a0 0x559d5381cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4132f89740 4 client.16074 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d53702800 con 0x559d533883a0 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4129d94640 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533883a0 0x559d5381cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f410d7fa640 10 client.16074 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.597+0000 7f4129d94640 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533883a0 0x559d5381cfb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ebfde64d25c953 server_cookie=e5db03c484406236 in_seq=0 out_seq=0 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f410d7fa640 1 -- 172.21.15.19:0/1091477763 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d53702800 con 0x559d533883a0 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f410d7fa640 10 client.16074 handle_command_reply: tid=0 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 client.16074 shutdown 2024-04-25T10:57:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 msgr2=0x559d537a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537a3ac0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f41140dc5e0 tx=0x7f41140076c0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 shutdown_connections 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d533883a0 0x559d5381cfb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d533f4390 0x559d537a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d533d36b0 0x559d537a3ac0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f410d7fa640 0 client.16074 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 --2- 172.21.15.19:0/1091477763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d533cb4d0 0x559d537a8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 >> 172.21.15.19:0/1091477763 conn(0x559d533e84e0 msgr2=0x559d537a2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 shutdown_connections 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 1 -- 172.21.15.19:0/1091477763 wait complete. 2024-04-25T10:57:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f4132f89740 20 client.16074 trim_cache size 0 max 16384 2024-04-25T10:57:47.606 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:57:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41100afce0 msgr2=0x7f41100b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f412bff8640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41100afce0 0x7f41100b21f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x559d53792980 tx=0x7f411c006f90 comp rx=0 tx=0).stop 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f412410b650 msgr2=0x7f412419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.598+0000 7f412bff8640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f412410b650 0x7f412419aca0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f411400c890 tx=0x7f411400cd50 comp rx=0 tx=0).stop 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 shutdown_connections 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f41100afce0 0x7f41100b21f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f412410e040 0x7f412419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f412410b650 0x7f412419aca0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 --2- 172.21.15.19:0/2463221901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124104bf0 0x7f412419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 >> 172.21.15.19:0/2463221901 conn(0x7f4124078100 msgr2=0x7f4124101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 shutdown_connections 2024-04-25T10:57:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:47.599+0000 7f412bff8640 1 -- 172.21.15.19:0/2463221901 wait complete. 2024-04-25T10:57:48.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:48 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3405197470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:48.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:48 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3405197470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:48.610 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:57:48.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:57:48.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:57:48.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:48 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3405197470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 -- 172.21.15.19:0/1138361743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff83010d380 msgr2=0x7ff83010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 --2- 172.21.15.19:0/1138361743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff83010d380 0x7ff83010f7a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff828009ca0 tx=0x7ff82802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 -- 172.21.15.19:0/1138361743 shutdown_connections 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 --2- 172.21.15.19:0/1138361743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff83010d380 0x7ff83010f7a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 --2- 172.21.15.19:0/1138361743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff830102af0 0x7ff830102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 --2- 172.21.15.19:0/1138361743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff830108a20 0x7ff8301025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 -- 172.21.15.19:0/1138361743 >> 172.21.15.19:0/1138361743 conn(0x7ff830078100 msgr2=0x7ff830078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 -- 172.21.15.19:0/1138361743 shutdown_connections 2024-04-25T10:57:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.741+0000 7ff836df3640 1 -- 172.21.15.19:0/1138361743 wait complete. 2024-04-25T10:57:48.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 Processor -- start 2024-04-25T10:57:48.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 -- start start 2024-04-25T10:57:48.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff830102af0 0x7ff830199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff830108a20 0x7ff830199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83010d380 0x7ff83019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff830112c00 con 0x7ff830102af0 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff830112a80 con 0x7ff830108a20 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff836df3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff830112d80 con 0x7ff83010d380 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff830102af0 0x7ff830199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff830102af0 0x7ff830199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35252/0 (socket says 172.21.15.19:35252) 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 -- 172.21.15.19:0/2325303821 learned_addr learned my addr 172.21.15.19:0/2325303821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff82ffff640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff830108a20 0x7ff830199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 -- 172.21.15.19:0/2325303821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83010d380 msgr2=0x7ff83019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83010d380 0x7ff83019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 -- 172.21.15.19:0/2325303821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff830108a20 msgr2=0x7ff830199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff830108a20 0x7ff830199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 -- 172.21.15.19:0/2325303821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83019e460 con 0x7ff830102af0 2024-04-25T10:57:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.742+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff830102af0 0x7ff830199600 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff82000d770 tx=0x7ff82000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.743+0000 7ff82dffb640 1 -- 172.21.15.19:0/2325303821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820017070 con 0x7ff830102af0 2024-04-25T10:57:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.743+0000 7ff82dffb640 1 -- 172.21.15.19:0/2325303821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff820009c30 con 0x7ff830102af0 2024-04-25T10:57:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.743+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83019e750 con 0x7ff830102af0 2024-04-25T10:57:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.743+0000 7ff82dffb640 1 -- 172.21.15.19:0/2325303821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82000f620 con 0x7ff830102af0 2024-04-25T10:57:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.743+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8301a60d0 con 0x7ff830102af0 2024-04-25T10:57:48.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff82dffb640 1 -- 172.21.15.19:0/2325303821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff82001a070 con 0x7ff830102af0 2024-04-25T10:57:48.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 Processor -- start 2024-04-25T10:57:48.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 -- start start 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f721430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f721970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 0x55b13f721eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b13f687b00 con 0x55b13f381410 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b13f623920 con 0x55b13f360730 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.746+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b13f683ac0 con 0x55b13f358550 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f721430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f721430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35262/0 (socket says 172.21.15.19:35262) 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 -- 172.21.15.19:0/2267202209 learned_addr learned my addr 172.21.15.19:0/2267202209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f721970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff82ffff640 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 0x55b13f721eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff82dffb640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff8180afce0 0x7ff8180b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 -- 172.21.15.19:0/2267202209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 msgr2=0x55b13f721970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f721970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 -- 172.21.15.19:0/2267202209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 msgr2=0x55b13f721eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 0x55b13f721eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 -- 172.21.15.19:0/2267202209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13f4617e0 con 0x55b13f381410 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff82dffb640 1 -- 172.21.15.19:0/2325303821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff8200deb40 con 0x7ff830102af0 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff8180afce0 0x7ff8180b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.747+0000 7ff835390640 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f721430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff8280097b0 tx=0x7ff828009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13c28fb04601323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.748+0000 7ff816ffd640 1 -- 172.21.15.19:0/2267202209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82804d070 con 0x55b13f381410 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.748+0000 7ff816ffd640 1 -- 172.21.15.19:0/2267202209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff828036040 con 0x55b13f381410 2024-04-25T10:57:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.748+0000 7ff816ffd640 1 -- 172.21.15.19:0/2267202209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82802f860 con 0x55b13f381410 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.748+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2267202209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 msgr2=0x55b13f721430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.748+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f721430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff8280097b0 tx=0x7ff828009800 comp rx=0 tx=0).stop 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.748+0000 7ff835390640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff8180afce0 0x7ff8180b21f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff82804a000 tx=0x7ff828047000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2267202209 shutdown_connections 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 0x55b13f721eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f721970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2267202209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f721430 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2267202209 >> 172.21.15.19:0/2267202209 conn(0x55b13f375560 msgr2=0x55b13f71ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2267202209 shutdown_connections 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2267202209 wait complete. 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 -- start start 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.749+0000 7ff83dd7f740 1 Processor -- start 2024-04-25T10:57:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f72ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f360730 0x55b13f72e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f381410 0x55b13f73a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b13f730b80 con 0x55b13f360730 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b13f6215d0 con 0x55b13f381410 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b13f730d00 con 0x55b13f358550 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff83dd7f740 10 client.0 _ll_register_callbacks cb 0x55b13f60f080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f360730 0x55b13f72e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f360730 0x55b13f72e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35272/0 (socket says 172.21.15.19:35272) 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 -- 172.21.15.19:0/2365507835 learned_addr learned my addr 172.21.15.19:0/2365507835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f72ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff835390640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f381410 0x55b13f73a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 -- 172.21.15.19:0/2365507835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 msgr2=0x55b13f72ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f72ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 -- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f381410 msgr2=0x55b13f73a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f381410 0x55b13f73a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff82ffff640 1 -- 172.21.15.19:0/2365507835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13f4c1f70 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f72ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.750+0000 7ff835390640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f381410 0x55b13f73a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff82ffff640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f360730 0x55b13f72e310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff82400be50 tx=0x7ff8240117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d2b75d62c1a7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff80ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff80ffff640 1 -- 172.21.15.19:0/2365507835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824004230 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff80ffff640 1 -- 172.21.15.19:0/2365507835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff8240043d0 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff828036040 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff80ffff640 1 -- 172.21.15.19:0/2365507835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff8240035f0 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 10 client.16086 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b13f359170 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff80ffff640 1 -- 172.21.15.19:0/2365507835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff828036040 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff80ffff640 1 -- 172.21.15.19:0/2365507835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff82401b8b0 con 0x55b13f360730 2024-04-25T10:57:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 10 client.16086 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 20 client.16086 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f3401f0 0x55b13f7402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 4 client.16086 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.751+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13f695710 con 0x55b13f3401f0 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.752+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f3401f0 0x55b13f7402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.752+0000 7ff80ffff640 10 client.16086 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.752+0000 7ff834b8f640 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f3401f0 0x55b13f7402b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b495e4d4e76feaff server_cookie=e046068f299cca85 in_seq=0 out_seq=0 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff80ffff640 1 -- 172.21.15.19:0/2365507835 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b13f695710 con 0x55b13f3401f0 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff80ffff640 10 client.16086 handle_command_reply: tid=0 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 client.16086 shutdown 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f360730 msgr2=0x55b13f72e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f360730 0x55b13f72e310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff82400be50 tx=0x7ff8240117b0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 shutdown_connections 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f3401f0 0x55b13f7402b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f381410 0x55b13f73a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f360730 0x55b13f72e310 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff80ffff640 0 client.16086 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/2365507835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f358550 0x55b13f72ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 >> 172.21.15.19:0/2365507835 conn(0x55b13f375560 msgr2=0x55b13f7232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 shutdown_connections 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 1 -- 172.21.15.19:0/2365507835 wait complete. 2024-04-25T10:57:48.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.753+0000 7ff83dd7f740 20 client.16086 trim_cache size 0 max 16384 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 Processor -- start 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 -- start start 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f358550 0x55b13f723cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 0x55b13f745a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f381410 0x55b13f745fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301ab5c0 con 0x55b13f358550 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301ade40 con 0x55b13f360730 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301abc30 con 0x55b13f381410 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff834b8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f358550 0x55b13f723cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff834b8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f358550 0x55b13f723cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35276/0 (socket says 172.21.15.19:35276) 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.772+0000 7ff834b8f640 1 -- 172.21.15.19:0/3191097709 learned_addr learned my addr 172.21.15.19:0/3191097709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff834b8f640 1 -- 172.21.15.19:0/3191097709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f381410 msgr2=0x55b13f745fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff834b8f640 1 --2- 172.21.15.19:0/3191097709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f381410 0x55b13f745fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff834b8f640 1 -- 172.21.15.19:0/3191097709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 msgr2=0x55b13f745a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff834b8f640 1 --2- 172.21.15.19:0/3191097709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 0x55b13f745a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff834b8f640 1 -- 172.21.15.19:0/3191097709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13f730460 con 0x55b13f358550 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff834b8f640 1 --2- 172.21.15.19:0/3191097709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f358550 0x55b13f723cb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8200dc8c0 tx=0x7ff8200c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9c094049da8158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff8167fc640 1 -- 172.21.15.19:0/3191097709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820021040 con 0x55b13f358550 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff8167fc640 1 -- 172.21.15.19:0/3191097709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8200a6820 con 0x55b13f358550 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.773+0000 7ff8167fc640 1 -- 172.21.15.19:0/3191097709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200a6d10 con 0x55b13f358550 2024-04-25T10:57:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3191097709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f358550 msgr2=0x55b13f723cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3191097709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f358550 0x55b13f723cb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8200dc8c0 tx=0x7ff8200c9040 comp rx=0 tx=0).stop 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3191097709 shutdown_connections 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3191097709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f381410 0x55b13f745fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3191097709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f360730 0x55b13f745a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3191097709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f358550 0x55b13f723cb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3191097709 >> 172.21.15.19:0/3191097709 conn(0x55b13f375560 msgr2=0x55b13f72cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3191097709 shutdown_connections 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3191097709 wait complete. 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 -- start start 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.774+0000 7ff83dd7f740 1 Processor -- start 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f358550 0x55b13f736470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f360730 0x55b13f7317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff83dd7f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f731d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301ac300 con 0x55b13f381410 2024-04-25T10:57:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301ad770 con 0x55b13f358550 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff83dd7f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301ac9d0 con 0x55b13f360730 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff83dd7f740 10 client.0 _ll_register_callbacks cb 0x7ff82400b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f731d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f731d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35290/0 (socket says 172.21.15.19:35290) 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 -- 172.21.15.19:0/3264035149 learned_addr learned my addr 172.21.15.19:0/3264035149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff834b8f640 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f358550 0x55b13f736470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff82ffff640 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f360730 0x55b13f7317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 -- 172.21.15.19:0/3264035149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f360730 msgr2=0x55b13f7317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f360730 0x55b13f7317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 -- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f358550 msgr2=0x55b13f736470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f358550 0x55b13f736470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 -- 172.21.15.19:0/3264035149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13f7323a0 con 0x55b13f381410 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff835390640 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f731d00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff82802f550 tx=0x7ff828009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3358470183db02e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff80ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.775+0000 7ff80ffff640 1 -- 172.21.15.19:0/3264035149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82804d030 con 0x55b13f381410 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff80ffff640 1 -- 172.21.15.19:0/3264035149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff8280305c0 con 0x55b13f381410 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8200a6820 con 0x55b13f381410 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff80ffff640 1 -- 172.21.15.19:0/3264035149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff8280318e0 con 0x55b13f381410 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 10 client.16092 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b13f7a84b0 con 0x55b13f381410 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff80ffff640 1 -- 172.21.15.19:0/3264035149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8200a6820 con 0x55b13f381410 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff80ffff640 1 -- 172.21.15.19:0/3264035149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff828044040 con 0x55b13f381410 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 10 client.16092 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 20 client.16092 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f315630 0x55b13f7aa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 4 client.16092 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.776+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b13f68f580 con 0x55b13f315630 2024-04-25T10:57:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.777+0000 7ff834b8f640 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f315630 0x55b13f7aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.777+0000 7ff834b8f640 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f315630 0x55b13f7aa830 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c050685b3ca8110f server_cookie=b56531c337ba3445 in_seq=0 out_seq=0 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.777+0000 7ff80ffff640 10 client.16092 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff80ffff640 1 -- 172.21.15.19:0/3264035149 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b13f68f580 con 0x55b13f315630 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff80ffff640 10 client.16092 handle_command_reply: tid=0 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 client.16092 shutdown 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 msgr2=0x55b13f731d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f731d00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff82802f550 tx=0x7ff828009440 comp rx=0 tx=0).stop 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 shutdown_connections 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55b13f315630 0x55b13f7aa830 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b13f381410 0x55b13f731d00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff80ffff640 0 client.16092 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b13f360730 0x55b13f7317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 --2- 172.21.15.19:0/3264035149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13f358550 0x55b13f736470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.778+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 >> 172.21.15.19:0/3264035149 conn(0x55b13f375560 msgr2=0x55b13f72f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 shutdown_connections 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff83dd7f740 1 -- 172.21.15.19:0/3264035149 wait complete. 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff83dd7f740 20 client.16092 trim_cache size 0 max 16384 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "7e5ce703-e1ec-44d9-83d5-65184757fe50", 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:57:48.784 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff8180afce0 msgr2=0x7ff8180b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff8180afce0 0x7ff8180b21f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff82804a000 tx=0x7ff828047000 comp rx=0 tx=0).stop 2024-04-25T10:57:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff830102af0 msgr2=0x7ff830199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff830102af0 0x7ff830199600 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff82000d770 tx=0x7ff82000dc30 comp rx=0 tx=0).stop 2024-04-25T10:57:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 shutdown_connections 2024-04-25T10:57:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff8180afce0 0x7ff8180b21f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83010d380 0x7ff83019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff830108a20 0x7ff830199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 --2- 172.21.15.19:0/2325303821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff830102af0 0x7ff830199600 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 >> 172.21.15.19:0/2325303821 conn(0x7ff830078100 msgr2=0x7ff8301040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 shutdown_connections 2024-04-25T10:57:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.779+0000 7ff836df3640 1 -- 172.21.15.19:0/2325303821 wait complete. 2024-04-25T10:57:48.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:57:48.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 -- 172.21.15.19:0/2957262621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20102720 msgr2=0x7f8a20110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 --2- 172.21.15.19:0/2957262621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20102720 0x7f8a20110570 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009ca0 tx=0x7f8a1802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 -- 172.21.15.19:0/2957262621 shutdown_connections 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 --2- 172.21.15.19:0/2957262621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20102720 0x7f8a20110570 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 --2- 172.21.15.19:0/2957262621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a20109f90 0x7f8a201020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 --2- 172.21.15.19:0/2957262621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a20109af0 0x7f8a20101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 -- 172.21.15.19:0/2957262621 >> 172.21.15.19:0/2957262621 conn(0x7f8a20077880 msgr2=0x7f8a20077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 -- 172.21.15.19:0/2957262621 shutdown_connections 2024-04-25T10:57:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.905+0000 7f8a254d1640 1 -- 172.21.15.19:0/2957262621 wait complete. 2024-04-25T10:57:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 Processor -- start 2024-04-25T10:57:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 -- start start 2024-04-25T10:57:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a20102720 0x7f8a2019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20109af0 0x7f8a2019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a20109f90 0x7f8a2019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20113d30 con 0x7f8a20109af0 2024-04-25T10:57:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20113bb0 con 0x7f8a20102720 2024-04-25T10:57:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a254d1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20113eb0 con 0x7f8a20109f90 2024-04-25T10:57:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20109af0 0x7f8a2019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20109af0 0x7f8a2019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35304/0 (socket says 172.21.15.19:35304) 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3245257655 learned_addr learned my addr 172.21.15.19:0/3245257655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a20109f90 0x7f8a2019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1effd640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a20102720 0x7f8a2019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3245257655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a20109f90 msgr2=0x7f8a2019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a20109f90 0x7f8a2019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3245257655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a20102720 msgr2=0x7f8a2019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a20102720 0x7f8a2019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3245257655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2019f4d0 con 0x7f8a20109af0 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.906+0000 7f8a1effd640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a20102720 0x7f8a2019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.907+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20109af0 0x7f8a2019abd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005fd0 tx=0x7f8a1400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.907+0000 7f8a07fff640 1 -- 172.21.15.19:0/3245257655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14017070 con 0x7f8a20109af0 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.907+0000 7f8a07fff640 1 -- 172.21.15.19:0/3245257655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a14009c30 con 0x7f8a20109af0 2024-04-25T10:57:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.907+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2019f7c0 con 0x7f8a20109af0 2024-04-25T10:57:48.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.907+0000 7f8a07fff640 1 -- 172.21.15.19:0/3245257655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003d30 con 0x7f8a20109af0 2024-04-25T10:57:48.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.907+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a201a7140 con 0x7f8a20109af0 2024-04-25T10:57:48.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.909+0000 7f8a07fff640 1 -- 172.21.15.19:0/3245257655 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8a14003ed0 con 0x7f8a20109af0 2024-04-25T10:57:48.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 Processor -- start 2024-04-25T10:57:48.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 -- start start 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc6098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6098fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55edc5ffed80 con 0x55edc5cd76b0 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edc5fb88b0 con 0x55edc5ccf4d0 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55edc5ffad40 con 0x55edc5cf8390 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a07fff640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8a080afce0 0x7f8a080b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc6098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a1effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6098fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc6098a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:46766/0 (socket says 172.21.15.19:46766) 2024-04-25T10:57:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/3249153432 learned_addr learned my addr 172.21.15.19:0/3249153432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35308/0 (socket says 172.21.15.19:35308) 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8a080afce0 0x7f8a080b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.910+0000 7f8a07fff640 1 -- 172.21.15.19:0/3245257655 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8a140decd0 con 0x7f8a20109af0 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/3249153432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 msgr2=0x55edc6098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6098fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/3249153432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 msgr2=0x55edc60a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/3249153432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edc5f89a40 con 0x55edc5cf8390 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1effd640 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc6098a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a18004af0 tx=0x7f8a180314d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999f3a8a62fe34a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8a080afce0 0x7f8a080b21f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009ca0 tx=0x7f8a18043000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a057fa640 1 -- 172.21.15.19:0/3249153432 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1804e070 con 0x55edc5cf8390 2024-04-25T10:57:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a057fa640 1 -- 172.21.15.19:0/3249153432 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a18036040 con 0x55edc5cf8390 2024-04-25T10:57:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.911+0000 7f8a057fa640 1 -- 172.21.15.19:0/3249153432 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18004670 con 0x55edc5cf8390 2024-04-25T10:57:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249153432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 msgr2=0x55edc6098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc6098a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a18004af0 tx=0x7f8a180314d0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249153432 shutdown_connections 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249153432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc6098a60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249153432 >> 172.21.15.19:0/3249153432 conn(0x55edc5cec4e0 msgr2=0x55edc6095210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249153432 shutdown_connections 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249153432 wait complete. 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 -- start start 2024-04-25T10:57:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.912+0000 7f8a2c37c740 1 Processor -- start 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5ccf4d0 0x55edc60a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cf8390 0x55edc60a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55edc60a5f80 con 0x55edc5cd76b0 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edc5f85d10 con 0x55edc5cf8390 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55edc60a6100 con 0x55edc5ccf4d0 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a2c37c740 10 client.0 _ll_register_callbacks cb 0x55edc5fb2210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cf8390 0x55edc60a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35324/0 (socket says 172.21.15.19:35324) 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cf8390 0x55edc60a94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:45070/0 (socket says 172.21.15.19:45070) 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3249954837 learned_addr learned my addr 172.21.15.19:0/3249954837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1effd640 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5ccf4d0 0x55edc60a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3249954837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5ccf4d0 msgr2=0x55edc60a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5ccf4d0 0x55edc60a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3249954837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cf8390 msgr2=0x55edc60a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cf8390 0x55edc60a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 -- 172.21.15.19:0/3249954837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edc5dda7d0 con 0x55edc5cd76b0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a3bd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8a140dc500 tx=0x7f8a140a6730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded27aed9dc2c7d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.913+0000 7f8a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a02ff5640 1 -- 172.21.15.19:0/3249954837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14009de0 con 0x55edc5cd76b0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a02ff5640 1 -- 172.21.15.19:0/3249954837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8a140dcba0 con 0x55edc5cd76b0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a18036040 con 0x55edc5cd76b0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a02ff5640 1 -- 172.21.15.19:0/3249954837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8a140a6ea0 con 0x55edc5cd76b0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 10 client.16104 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55edc5d103f0 con 0x55edc5cd76b0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a02ff5640 1 -- 172.21.15.19:0/3249954837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a18036040 con 0x55edc5cd76b0 2024-04-25T10:57:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a02ff5640 1 -- 172.21.15.19:0/3249954837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8a140bd070 con 0x55edc5cd76b0 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 10 client.16104 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 20 client.16104 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5cb6f60 0x55edc60b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 4 client.16104 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edc600c990 con 0x55edc5cb6f60 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.914+0000 7f8a1effd640 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5cb6f60 0x55edc60b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.915+0000 7f8a02ff5640 10 client.16104 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.915+0000 7f8a1effd640 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5cb6f60 0x55edc60b57c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85adc3700f3f230a server_cookie=aee5ed0db6c72efe in_seq=0 out_seq=0 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a02ff5640 1 -- 172.21.15.19:0/3249954837 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55edc600c990 con 0x55edc5cb6f60 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a02ff5640 10 client.16104 handle_command_reply: tid=0 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 client.16104 shutdown 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 msgr2=0x55edc60a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a3bd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8a140dc500 tx=0x7f8a140a6730 comp rx=0 tx=0).stop 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 shutdown_connections 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5cb6f60 0x55edc60b57c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cf8390 0x55edc60a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a02ff5640 0 client.16104 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cd76b0 0x55edc60a3bd0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/3249954837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5ccf4d0 0x55edc60a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 >> 172.21.15.19:0/3249954837 conn(0x55edc5cec4e0 msgr2=0x55edc609a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 shutdown_connections 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 1 -- 172.21.15.19:0/3249954837 wait complete. 2024-04-25T10:57:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.916+0000 7f8a2c37c740 20 client.16104 trim_cache size 0 max 16384 2024-04-25T10:57:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 Processor -- start 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 -- start start 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5ccf4d0 0x55edc60a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cd76b0 0x55edc6098f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc60baef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201ad920 con 0x55edc5ccf4d0 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201b01a0 con 0x55edc5cd76b0 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201adf90 con 0x55edc5cf8390 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a1effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5ccf4d0 0x55edc60a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a1effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5ccf4d0 0x55edc60a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35332/0 (socket says 172.21.15.19:35332) 2024-04-25T10:57:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a1effd640 1 -- 172.21.15.19:0/1485555108 learned_addr learned my addr 172.21.15.19:0/1485555108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.934+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cd76b0 0x55edc6098f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc60baef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1effd640 1 -- 172.21.15.19:0/1485555108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 msgr2=0x55edc60baef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1effd640 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc60baef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1effd640 1 -- 172.21.15.19:0/1485555108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cd76b0 msgr2=0x55edc6098f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1effd640 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cd76b0 0x55edc6098f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1effd640 1 -- 172.21.15.19:0/1485555108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edc5cd00f0 con 0x55edc5ccf4d0 2024-04-25T10:57:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc60baef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cd76b0 0x55edc6098f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a1effd640 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5ccf4d0 0x55edc60a23c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55edc609a230 tx=0x7f8a100094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc26e918cc9c2f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a04ff9640 1 -- 172.21.15.19:0/1485555108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1001f070 con 0x55edc5ccf4d0 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a04ff9640 1 -- 172.21.15.19:0/1485555108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a10004730 con 0x55edc5ccf4d0 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.935+0000 7f8a04ff9640 1 -- 172.21.15.19:0/1485555108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1001b5c0 con 0x55edc5ccf4d0 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1485555108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5ccf4d0 msgr2=0x55edc60a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5ccf4d0 0x55edc60a23c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55edc609a230 tx=0x7f8a100094a0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1485555108 shutdown_connections 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cf8390 0x55edc60baef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5cd76b0 0x55edc6098f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1485555108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5ccf4d0 0x55edc60a23c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1485555108 >> 172.21.15.19:0/1485555108 conn(0x55edc5cec4e0 msgr2=0x55edc60987f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1485555108 shutdown_connections 2024-04-25T10:57:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1485555108 wait complete. 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- start start 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 Processor -- start 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cd76b0 0x55edc6116300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cf8390 0x55edc60aae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201ae660 con 0x55edc5cf8390 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201afad0 con 0x55edc5ccf4d0 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201aed30 con 0x55edc5cd76b0 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.936+0000 7f8a2c37c740 10 client.0 _ll_register_callbacks cb 0x7f8a140a2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:57:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cf8390 0x55edc60aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cf8390 0x55edc60aae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35342/0 (socket says 172.21.15.19:35342) 2024-04-25T10:57:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/1566761403 learned_addr learned my addr 172.21.15.19:0/1566761403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:57:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cd76b0 0x55edc6116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1effd640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/1566761403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cd76b0 msgr2=0x55edc6116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cd76b0 0x55edc6116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 msgr2=0x55edc6115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 -- 172.21.15.19:0/1566761403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edc5fbf4b0 con 0x55edc5cf8390 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1e7fc640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cd76b0 0x55edc6116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1effd640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a1f7fe640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cf8390 0x55edc60aae30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8a1802a900 tx=0x7f8a18009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21913d8306767699 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a02ff5640 1 -- 172.21.15.19:0/1566761403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1804e070 con 0x55edc5cf8390 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a02ff5640 1 -- 172.21.15.19:0/1566761403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8a18003e00 con 0x55edc5cf8390 2024-04-25T10:57:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a10004730 con 0x55edc5cf8390 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a02ff5640 1 -- 172.21.15.19:0/1566761403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8a18061050 con 0x55edc5cf8390 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a2c37c740 10 client.16110 fetch_fsmap learned FSMap version 20 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.937+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55edc611d1d0 con 0x55edc5cf8390 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a02ff5640 1 -- 172.21.15.19:0/1566761403 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a10004730 con 0x55edc5cf8390 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a02ff5640 1 -- 172.21.15.19:0/1566761403 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8a1802fbe0 con 0x55edc5cf8390 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a2c37c740 10 client.16110 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a2c37c740 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a2c37c740 20 client.16110 populate_metadata read hostname 'smithi019' 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5c8c3a0 0x55edc611f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a2c37c740 4 client.16110 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55edc6006800 con 0x55edc5c8c3a0 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a1effd640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5c8c3a0 0x55edc611f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:57:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a1effd640 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5c8c3a0 0x55edc611f510 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55cf9ba28aa88317 server_cookie=87d7a0859b7b0df9 in_seq=0 out_seq=0 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.938+0000 7f8a02ff5640 10 client.16110 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a02ff5640 1 -- 172.21.15.19:0/1566761403 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55edc6006800 con 0x55edc5c8c3a0 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a02ff5640 10 client.16110 handle_command_reply: tid=0 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 client.16110 shutdown 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cf8390 msgr2=0x55edc60aae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cf8390 0x55edc60aae30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8a1802a900 tx=0x7f8a18009b60 comp rx=0 tx=0).stop 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 shutdown_connections 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55edc5c8c3a0 0x55edc611f510 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55edc5cf8390 0x55edc60aae30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55edc5cd76b0 0x55edc6116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 --2- 172.21.15.19:0/1566761403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edc5ccf4d0 0x55edc6115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a02ff5640 0 client.16110 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 >> 172.21.15.19:0/1566761403 conn(0x55edc5cec4e0 msgr2=0x55edc609a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 shutdown_connections 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 1 -- 172.21.15.19:0/1566761403 wait complete. 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.939+0000 7f8a2c37c740 20 client.16110 trim_cache size 0 max 16384 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stdout: "7e5ce703-e1ec-44d9-83d5-65184757fe50": { 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:57:48.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "7e5ce703-e1ec-44d9-83d5-65184757fe50", 2024-04-25T10:57:48.950 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:57:48.950 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:57:48.950 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:57:48.950 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8a080afce0 msgr2=0x7f8a080b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8a080afce0 0x7f8a080b21f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009ca0 tx=0x7f8a18043000 comp rx=0 tx=0).stop 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20109af0 msgr2=0x7f8a2019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20109af0 0x7f8a2019abd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005fd0 tx=0x7f8a1400dde0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 shutdown_connections 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8a080afce0 0x7f8a080b21f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a20109f90 0x7f8a2019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20109af0 0x7f8a2019abd0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 --2- 172.21.15.19:0/3245257655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a20102720 0x7f8a2019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 >> 172.21.15.19:0/3245257655 conn(0x7f8a20077880 msgr2=0x7f8a201004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:57:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 shutdown_connections 2024-04-25T10:57:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:57:48.940+0000 7f8a254d1640 1 -- 172.21.15.19:0/3245257655 wait complete. 2024-04-25T10:57:48.952 INFO:tasks.cephfs.filesystem:scrub status for tag:7e5ce703-e1ec-44d9-83d5-65184757fe50 - {'path': '/', 'tag': '7e5ce703-e1ec-44d9-83d5-65184757fe50', 'options': 'recursive,force'} 2024-04-25T10:57:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:49 smithi019 ceph-mon[29209]: pgmap v102: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:49 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:57:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:49 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:49 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:57:49.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:49 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:49 smithi169 ceph-mon[33049]: pgmap v102: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:49 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:57:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:49 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:49 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:57:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:49 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:49 smithi112 ceph-mon[32799]: pgmap v102: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:49 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:57:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:49 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:49 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:57:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:49 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:57:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:51 smithi169 ceph-mon[33049]: pgmap v103: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:51.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:51 smithi112 ceph-mon[32799]: pgmap v103: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:51.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:51 smithi019 ceph-mon[29209]: pgmap v103: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:57:52.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:52 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:57:52.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:52 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:57:52.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:52 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:57:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:53 smithi169 ceph-mon[33049]: pgmap v104: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-04-25T10:57:53.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:53 smithi112 ceph-mon[32799]: pgmap v104: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-04-25T10:57:53.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:53 smithi019 ceph-mon[29209]: pgmap v104: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-04-25T10:57:55.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:55 smithi169 ceph-mon[33049]: pgmap v105: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:57:55.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:55 smithi112 ceph-mon[32799]: pgmap v105: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:57:55.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:55 smithi019 ceph-mon[29209]: pgmap v105: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:57:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:57 smithi169 ceph-mon[33049]: pgmap v106: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-04-25T10:57:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:57 smithi112 ceph-mon[32799]: pgmap v106: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-04-25T10:57:57.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:57 smithi019 ceph-mon[29209]: pgmap v106: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-04-25T10:57:59.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:57:59 smithi169 ceph-mon[33049]: pgmap v107: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:57:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:57:59 smithi112 ceph-mon[32799]: pgmap v107: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:57:59.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:57:59 smithi019 ceph-mon[29209]: pgmap v107: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:58:01.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:01 smithi169 ceph-mon[33049]: pgmap v108: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:58:01.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:01 smithi112 ceph-mon[32799]: pgmap v108: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:58:01.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:01 smithi019 ceph-mon[29209]: pgmap v108: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-04-25T10:58:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:03 smithi169 ceph-mon[33049]: pgmap v109: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-25T10:58:03.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:03 smithi112 ceph-mon[32799]: pgmap v109: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-25T10:58:03.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:03 smithi019 ceph-mon[29209]: pgmap v109: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-25T10:58:05.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:05 smithi169 ceph-mon[33049]: pgmap v110: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:05.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:05 smithi112 ceph-mon[32799]: pgmap v110: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:05.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:05 smithi019 ceph-mon[29209]: pgmap v110: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:07 smithi169 ceph-mon[33049]: pgmap v111: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:58:07.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:07 smithi112 ceph-mon[32799]: pgmap v111: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:58:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:07 smithi019 ceph-mon[29209]: pgmap v111: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:58:09.502 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:09 smithi169 ceph-mon[33049]: pgmap v112: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:09.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:09 smithi112 ceph-mon[32799]: pgmap v112: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:09.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:09 smithi019 ceph-mon[29209]: pgmap v112: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:11.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:11 smithi169 ceph-mon[33049]: pgmap v113: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:11.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:11 smithi112 ceph-mon[32799]: pgmap v113: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:11.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:11 smithi019 ceph-mon[29209]: pgmap v113: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:58:13.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:13 smithi169 ceph-mon[33049]: pgmap v114: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:58:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:13 smithi112 ceph-mon[32799]: pgmap v114: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:58:13.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:13 smithi019 ceph-mon[29209]: pgmap v114: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:58:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:15 smithi169 ceph-mon[33049]: pgmap v115: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T10:58:15.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:15 smithi112 ceph-mon[32799]: pgmap v115: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T10:58:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:15 smithi019 ceph-mon[29209]: pgmap v115: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T10:58:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:17 smithi169 ceph-mon[33049]: pgmap v116: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:58:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:17 smithi112 ceph-mon[32799]: pgmap v116: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:58:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:17 smithi019 ceph-mon[29209]: pgmap v116: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:58:18.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:58:18.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:58:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 -- 172.21.15.19:0/1897538455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee401027e0 msgr2=0x7fee40110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 --2- 172.21.15.19:0/1897538455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee401027e0 0x7fee40110690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fee38009ca0 tx=0x7fee3802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 -- 172.21.15.19:0/1897538455 shutdown_connections 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 --2- 172.21.15.19:0/1897538455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee401027e0 0x7fee40110690 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 --2- 172.21.15.19:0/1897538455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4010a040 0x7fee40102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 --2- 172.21.15.19:0/1897538455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee40109ba0 0x7fee40101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 -- 172.21.15.19:0/1897538455 >> 172.21.15.19:0/1897538455 conn(0x7fee40077880 msgr2=0x7fee40077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 -- 172.21.15.19:0/1897538455 shutdown_connections 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 -- 172.21.15.19:0/1897538455 wait complete. 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 Processor -- start 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.082+0000 7fee47306640 1 -- start start 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee47306640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee401027e0 0x7fee4019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee47306640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40109ba0 0x7fee4019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee47306640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4010a040 0x7fee4019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee47306640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee40113d60 con 0x7fee40109ba0 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee47306640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee40113be0 con 0x7fee401027e0 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee47306640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee40113ee0 con 0x7fee4010a040 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40109ba0 0x7fee4019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee450a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee401027e0 0x7fee4019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40109ba0 0x7fee4019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42504/0 (socket says 172.21.15.19:42504) 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 -- 172.21.15.19:0/3008462659 learned_addr learned my addr 172.21.15.19:0/3008462659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee458a3640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4010a040 0x7fee4019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 -- 172.21.15.19:0/3008462659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4010a040 msgr2=0x7fee4019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4010a040 0x7fee4019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 -- 172.21.15.19:0/3008462659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee401027e0 msgr2=0x7fee4019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee401027e0 0x7fee4019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.083+0000 7fee448a1640 1 -- 172.21.15.19:0/3008462659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4019f560 con 0x7fee40109ba0 2024-04-25T10:58:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.084+0000 7fee448a1640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40109ba0 0x7fee4019ace0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c9a0 tx=0x7fee3000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.084+0000 7fee367fc640 1 -- 172.21.15.19:0/3008462659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30014070 con 0x7fee40109ba0 2024-04-25T10:58:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.084+0000 7fee367fc640 1 -- 172.21.15.19:0/3008462659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee30004440 con 0x7fee40109ba0 2024-04-25T10:58:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.084+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4019f850 con 0x7fee40109ba0 2024-04-25T10:58:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.084+0000 7fee367fc640 1 -- 172.21.15.19:0/3008462659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3000f450 con 0x7fee40109ba0 2024-04-25T10:58:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.084+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4019fda0 con 0x7fee40109ba0 2024-04-25T10:58:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.087+0000 7fee4e1ba740 1 Processor -- start 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.087+0000 7fee4e1ba740 1 -- start start 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.087+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df73978a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df73978fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df739858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df738ded80 con 0x55df735af4d0 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df738988b0 con 0x55df735b76b0 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df738dad40 con 0x55df735d8390 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee367fc640 1 -- 172.21.15.19:0/3008462659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fee30017070 con 0x7fee40109ba0 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df73978fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df73978fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42518/0 (socket says 172.21.15.19:42518) 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 -- 172.21.15.19:0/3676757236 learned_addr learned my addr 172.21.15.19:0/3676757236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee458a3640 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df73978a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee448a1640 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df739858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 -- 172.21.15.19:0/3676757236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 msgr2=0x55df73978a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df73978a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 -- 172.21.15.19:0/3676757236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 msgr2=0x55df739858f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df739858f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 -- 172.21.15.19:0/3676757236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df73869a40 con 0x55df735af4d0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee448a1640 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df739858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.088+0000 7fee450a2640 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df73978fa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00bd10 tx=0x7fee3c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f046421b682b94c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.089+0000 7fee2b7fe640 1 -- 172.21.15.19:0/3676757236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c004030 con 0x55df735af4d0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.089+0000 7fee2b7fe640 1 -- 172.21.15.19:0/3676757236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0035f0 con 0x55df735af4d0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.089+0000 7fee2b7fe640 1 -- 172.21.15.19:0/3676757236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011770 con 0x55df735af4d0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.089+0000 7fee367fc640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee2c0afc60 0x7fee2c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.089+0000 7fee458a3640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee2c0afc60 0x7fee2c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.089+0000 7fee367fc640 1 -- 172.21.15.19:0/3008462659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fee300dea20 con 0x7fee40109ba0 2024-04-25T10:58:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3676757236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 msgr2=0x55df73978fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee458a3640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee2c0afc60 0x7fee2c0b2170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55df73979b60 tx=0x7fee3802f310 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df73978fa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00bd10 tx=0x7fee3c0107a0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3676757236 shutdown_connections 2024-04-25T10:58:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df739858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df73978fa0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3676757236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df73978a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3676757236 >> 172.21.15.19:0/3676757236 conn(0x55df735cc4e0 msgr2=0x55df73975210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3676757236 shutdown_connections 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3676757236 wait complete. 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 -- start start 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.090+0000 7fee4e1ba740 1 Processor -- start 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735af4d0 0x55df739835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df73983b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735d8390 0x55df73989430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df73985fe0 con 0x55df735d8390 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df73865d10 con 0x55df735b76b0 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df73986160 con 0x55df735af4d0 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee4e1ba740 10 client.0 _ll_register_callbacks cb 0x55df73892210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee458a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735d8390 0x55df73989430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee458a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735d8390 0x55df73989430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42526/0 (socket says 172.21.15.19:42526) 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee458a3640 1 -- 172.21.15.19:0/1302706515 learned_addr learned my addr 172.21.15.19:0/1302706515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee448a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df73983b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee450a2640 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735af4d0 0x55df739835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee448a1640 1 -- 172.21.15.19:0/1302706515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735af4d0 msgr2=0x55df739835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee448a1640 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735af4d0 0x55df739835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee448a1640 1 -- 172.21.15.19:0/1302706515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735d8390 msgr2=0x55df73989430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee448a1640 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735d8390 0x55df73989430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee448a1640 1 -- 172.21.15.19:0/1302706515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df736ba7d0 con 0x55df735b76b0 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.091+0000 7fee458a3640 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735d8390 0x55df73989430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee448a1640 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df73983b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee300dc1a0 tx=0x7fee30000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee6f5fd1ea748e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee28ff9640 1 -- 172.21.15.19:0/1302706515 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee300a6540 con 0x55df735b76b0 2024-04-25T10:58:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c0035f0 con 0x55df735b76b0 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee28ff9640 1 -- 172.21.15.19:0/1302706515 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fee300a5e50 con 0x55df735b76b0 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee28ff9640 1 -- 172.21.15.19:0/1302706515 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fee300a6360 con 0x55df735b76b0 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee4e1ba740 10 client.25718 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55df735f03f0 con 0x55df735b76b0 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.092+0000 7fee28ff9640 1 -- 172.21.15.19:0/1302706515 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0035f0 con 0x55df735b76b0 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee28ff9640 1 -- 172.21.15.19:0/1302706515 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fee300a2560 con 0x55df735b76b0 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee4e1ba740 10 client.25718 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee4e1ba740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee4e1ba740 20 client.25718 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df73596f60 0x55df73995690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee4e1ba740 4 client.25718 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df738ec990 con 0x55df73596f60 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.093+0000 7fee450a2640 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df73596f60 0x55df73995690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.094+0000 7fee450a2640 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df73596f60 0x55df73995690 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a195c7f85ab5e5f server_cookie=4e36c63da3481d1a in_seq=0 out_seq=0 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.094+0000 7fee28ff9640 10 client.25718 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee28ff9640 1 -- 172.21.15.19:0/1302706515 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df738ec990 con 0x55df73596f60 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee28ff9640 10 client.25718 handle_command_reply: tid=0 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 client.25718 shutdown 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 msgr2=0x55df73983b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df73983b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee300dc1a0 tx=0x7fee30000bc0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 shutdown_connections 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df73596f60 0x55df73995690 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735d8390 0x55df73989430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735b76b0 0x55df73983b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1302706515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735af4d0 0x55df739835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee28ff9640 0 client.25718 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 >> 172.21.15.19:0/1302706515 conn(0x55df735cc4e0 msgr2=0x55df7397a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 shutdown_connections 2024-04-25T10:58:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.095+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1302706515 wait complete. 2024-04-25T10:58:19.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.096+0000 7fee4e1ba740 20 client.25718 trim_cache size 0 max 16384 2024-04-25T10:58:19.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 Processor -- start 2024-04-25T10:58:19.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 -- start start 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df7397ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735b76b0 0x55df7397adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735d8390 0x55df73989420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee401ad8e0 con 0x55df735af4d0 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee401b0160 con 0x55df735d8390 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee401adf50 con 0x55df735b76b0 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df7397ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df7397ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42540/0 (socket says 172.21.15.19:42540) 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 -- 172.21.15.19:0/1219426166 learned_addr learned my addr 172.21.15.19:0/1219426166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee458a3640 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735d8390 0x55df73989420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee448a1640 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735b76b0 0x55df7397adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 -- 172.21.15.19:0/1219426166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735b76b0 msgr2=0x55df7397adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735b76b0 0x55df7397adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 -- 172.21.15.19:0/1219426166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735d8390 msgr2=0x55df73989420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735d8390 0x55df73989420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 -- 172.21.15.19:0/1219426166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df735b00f0 con 0x55df735af4d0 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee448a1640 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735b76b0 0x55df7397adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.114+0000 7fee450a2640 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df7397ebe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fee3c010ca0 tx=0x7fee3c003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e559eb91cf8d49f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee2affd640 1 -- 172.21.15.19:0/1219426166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0037f0 con 0x55df735af4d0 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee2affd640 1 -- 172.21.15.19:0/1219426166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0054f0 con 0x55df735af4d0 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee2affd640 1 -- 172.21.15.19:0/1219426166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c029450 con 0x55df735af4d0 2024-04-25T10:58:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1219426166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 msgr2=0x55df7397ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df7397ebe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fee3c010ca0 tx=0x7fee3c003d20 comp rx=0 tx=0).stop 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1219426166 shutdown_connections 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735d8390 0x55df73989420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735b76b0 0x55df7397adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/1219426166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735af4d0 0x55df7397ebe0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1219426166 >> 172.21.15.19:0/1219426166 conn(0x55df735cc4e0 msgr2=0x55df7397a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1219426166 shutdown_connections 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.115+0000 7fee4e1ba740 1 -- 172.21.15.19:0/1219426166 wait complete. 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 -- start start 2024-04-25T10:58:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 Processor -- start 2024-04-25T10:58:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735af4d0 0x55df739f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735b76b0 0x55df739f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df7398aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee401ae620 con 0x55df735b76b0 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee401afa90 con 0x55df735af4d0 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee401aecf0 con 0x55df735d8390 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee4e1ba740 10 client.0 _ll_register_callbacks cb 0x7fee300e3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee448a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735b76b0 0x55df739f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee448a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735b76b0 0x55df739f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42550/0 (socket says 172.21.15.19:42550) 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.116+0000 7fee448a1640 1 -- 172.21.15.19:0/3692392499 learned_addr learned my addr 172.21.15.19:0/3692392499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee458a3640 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df7398aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee450a2640 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735af4d0 0x55df739f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee448a1640 1 -- 172.21.15.19:0/3692392499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 msgr2=0x55df7398aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee448a1640 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df7398aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee448a1640 1 -- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735af4d0 msgr2=0x55df739f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee448a1640 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735af4d0 0x55df739f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee448a1640 1 -- 172.21.15.19:0/3692392499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7389f4b0 con 0x55df735b76b0 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee448a1640 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735b76b0 0x55df739f6260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fee300a3e60 tx=0x7fee3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbc8957a8cca7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee28ff9640 1 -- 172.21.15.19:0/3692392499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee300a6540 con 0x55df735b76b0 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee28ff9640 1 -- 172.21.15.19:0/3692392499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fee300a5ad0 con 0x55df735b76b0 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c0054f0 con 0x55df735b76b0 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee28ff9640 1 -- 172.21.15.19:0/3692392499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fee300a2ab0 con 0x55df735b76b0 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee4e1ba740 10 client.16128 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55df739fd130 con 0x55df735b76b0 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.117+0000 7fee28ff9640 1 -- 172.21.15.19:0/3692392499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0054f0 con 0x55df735b76b0 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee28ff9640 1 -- 172.21.15.19:0/3692392499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fee300c90b0 con 0x55df735b76b0 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee4e1ba740 10 client.16128 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee4e1ba740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee4e1ba740 20 client.16128 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df7356c3a0 0x55df739ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee4e1ba740 4 client.16128 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df738e6800 con 0x55df7356c3a0 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee450a2640 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df7356c3a0 0x55df739ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee450a2640 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df7356c3a0 0x55df739ff7b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfb3f56a80f11e server_cookie=1c1e86703f9ecc84 in_seq=0 out_seq=0 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.118+0000 7fee28ff9640 10 client.16128 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee28ff9640 1 -- 172.21.15.19:0/3692392499 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df738e6800 con 0x55df7356c3a0 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee28ff9640 10 client.16128 handle_command_reply: tid=0 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 client.16128 shutdown 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735b76b0 msgr2=0x55df739f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735b76b0 0x55df739f6260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fee300a3e60 tx=0x7fee3000cf90 comp rx=0 tx=0).stop 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 shutdown_connections 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55df7356c3a0 0x55df739ff7b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df735d8390 0x55df7398aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee28ff9640 0 client.16128 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df735b76b0 0x55df739f6260 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 --2- 172.21.15.19:0/3692392499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df735af4d0 0x55df739f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 >> 172.21.15.19:0/3692392499 conn(0x55df735cc4e0 msgr2=0x55df73984a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 shutdown_connections 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 1 -- 172.21.15.19:0/3692392499 wait complete. 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee4e1ba740 20 client.16128 trim_cache size 0 max 16384 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:58:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee2c0afc60 msgr2=0x7fee2c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee47306640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee2c0afc60 0x7fee2c0b2170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55df73979b60 tx=0x7fee3802f310 comp rx=0 tx=0).stop 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40109ba0 msgr2=0x7fee4019ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.119+0000 7fee47306640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40109ba0 0x7fee4019ace0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c9a0 tx=0x7fee3000ce60 comp rx=0 tx=0).stop 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 shutdown_connections 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fee2c0afc60 0x7fee2c0b2170 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4010a040 0x7fee4019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40109ba0 0x7fee4019ace0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 --2- 172.21.15.19:0/3008462659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee401027e0 0x7fee4019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 >> 172.21.15.19:0/3008462659 conn(0x7fee40077880 msgr2=0x7fee400ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 shutdown_connections 2024-04-25T10:58:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.120+0000 7fee47306640 1 -- 172.21.15.19:0/3008462659 wait complete. 2024-04-25T10:58:19.131 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:58:19.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 -- 172.21.15.19:0/3748936221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260109fb0 msgr2=0x7fe260103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 --2- 172.21.15.19:0/3748936221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260109fb0 0x7fe260103c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe254009900 tx=0x7fe25402a410 comp rx=0 tx=0).stop 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 -- 172.21.15.19:0/3748936221 shutdown_connections 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 --2- 172.21.15.19:0/3748936221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2601042b0 0x7fe260110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 --2- 172.21.15.19:0/3748936221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260109fb0 0x7fe260103c40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 --2- 172.21.15.19:0/3748936221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260109b10 0x7fe260103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 -- 172.21.15.19:0/3748936221 >> 172.21.15.19:0/3748936221 conn(0x7fe2600ff360 msgr2=0x7fe260101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 -- 172.21.15.19:0/3748936221 shutdown_connections 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.246+0000 7fe267249640 1 -- 172.21.15.19:0/3748936221 wait complete. 2024-04-25T10:58:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 Processor -- start 2024-04-25T10:58:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 -- start start 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2601042b0 0x7fe26019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe260109b10 0x7fe26019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260109fb0 0x7fe2601a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe260113d00 con 0x7fe2601042b0 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe260113b80 con 0x7fe260109fb0 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe267249640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe260113e80 con 0x7fe260109b10 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2601042b0 0x7fe26019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2601042b0 0x7fe26019d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42566/0 (socket says 172.21.15.19:42566) 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 -- 172.21.15.19:0/2292761219 learned_addr learned my addr 172.21.15.19:0/2292761219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe2657e6640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260109fb0 0x7fe2601a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe25ffff640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe260109b10 0x7fe26019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 -- 172.21.15.19:0/2292761219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe260109b10 msgr2=0x7fe26019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe260109b10 0x7fe26019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 -- 172.21.15.19:0/2292761219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260109fb0 msgr2=0x7fe2601a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260109fb0 0x7fe2601a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe264fe5640 1 -- 172.21.15.19:0/2292761219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2601a8b80 con 0x7fe2601042b0 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe25ffff640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe260109b10 0x7fe26019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.247+0000 7fe2657e6640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260109fb0 0x7fe2601a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.248+0000 7fe264fe5640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2601042b0 0x7fe26019d7a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe25800eb90 tx=0x7fe25800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.248+0000 7fe25dffb640 1 -- 172.21.15.19:0/2292761219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258010630 con 0x7fe2601042b0 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.248+0000 7fe25dffb640 1 -- 172.21.15.19:0/2292761219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25800ee90 con 0x7fe2601042b0 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.248+0000 7fe25dffb640 1 -- 172.21.15.19:0/2292761219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258018ac0 con 0x7fe2601042b0 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.248+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2601a8d50 con 0x7fe2601042b0 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.248+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601a9240 con 0x7fe2601042b0 2024-04-25T10:58:19.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.249+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26006afa0 con 0x7fe2601042b0 2024-04-25T10:58:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.250+0000 7fe25dffb640 1 -- 172.21.15.19:0/2292761219 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe258018c60 con 0x7fe2601042b0 2024-04-25T10:58:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.251+0000 7fe25dffb640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2480afc60 0x7fe2480b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.251+0000 7fe25ffff640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2480afc60 0x7fe2480b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.251+0000 7fe25dffb640 1 -- 172.21.15.19:0/2292761219 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe2580dfb30 con 0x7fe2601042b0 2024-04-25T10:58:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.252+0000 7fe25ffff640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2480afc60 0x7fe2480b2170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe254031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.255+0000 7fe25dffb640 1 -- 172.21.15.19:0/2292761219 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2580a8400 con 0x7fe2601042b0 2024-04-25T10:58:19.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:19 smithi019 ceph-mon[29209]: pgmap v117: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:58:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.481+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2601a95a0 con 0x7fe2601042b0 2024-04-25T10:58:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.482+0000 7fe25dffb640 1 -- 172.21.15.19:0/2292761219 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fe2580a7d00 con 0x7fe2601042b0 2024-04-25T10:58:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:58:19.482 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:58:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:58:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2480afc60 msgr2=0x7fe2480b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2480afc60 0x7fe2480b2170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe254031400 comp rx=0 tx=0).stop 2024-04-25T10:58:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2601042b0 msgr2=0x7fe26019d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2601042b0 0x7fe26019d7a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe25800eb90 tx=0x7fe25800c6a0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 shutdown_connections 2024-04-25T10:58:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2480afc60 0x7fe2480b2170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260109fb0 0x7fe2601a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe260109b10 0x7fe26019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 --2- 172.21.15.19:0/2292761219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2601042b0 0x7fe26019d7a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 >> 172.21.15.19:0/2292761219 conn(0x7fe2600ff360 msgr2=0x7fe2600754a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 shutdown_connections 2024-04-25T10:58:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.484+0000 7fe267249640 1 -- 172.21.15.19:0/2292761219 wait complete. 2024-04-25T10:58:19.496 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:58:19.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:19 smithi169 ceph-mon[33049]: pgmap v117: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:58:19.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:19 smithi112 ceph-mon[32799]: pgmap v117: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 -- 172.21.15.19:0/3435913502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f10102af0 msgr2=0x7f8f10102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 --2- 172.21.15.19:0/3435913502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f10102af0 0x7f8f10102f00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009900 tx=0x7f8f0402a410 comp rx=0 tx=0).stop 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 -- 172.21.15.19:0/3435913502 shutdown_connections 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 --2- 172.21.15.19:0/3435913502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1010d380 0x7f8f1010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 --2- 172.21.15.19:0/3435913502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f10102af0 0x7f8f10102f00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 --2- 172.21.15.19:0/3435913502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f10108a20 0x7f8f101025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 -- 172.21.15.19:0/3435913502 >> 172.21.15.19:0/3435913502 conn(0x7f8f10078100 msgr2=0x7f8f10078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 -- 172.21.15.19:0/3435913502 shutdown_connections 2024-04-25T10:58:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.610+0000 7f8f15a70640 1 -- 172.21.15.19:0/3435913502 wait complete. 2024-04-25T10:58:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 Processor -- start 2024-04-25T10:58:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 -- start start 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f10102af0 0x7f8f1019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f10108a20 0x7f8f1019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1010d380 0x7f8f101a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10112be0 con 0x7f8f1010d380 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10112a60 con 0x7f8f10102af0 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f15a70640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10112d60 con 0x7f8f10108a20 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1010d380 0x7f8f101a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1010d380 0x7f8f101a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42586/0 (socket says 172.21.15.19:42586) 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f0ffff640 1 -- 172.21.15.19:0/336986830 learned_addr learned my addr 172.21.15.19:0/336986830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.611+0000 7f8f0effd640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f10108a20 0x7f8f1019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f10102af0 0x7f8f1019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0ffff640 1 -- 172.21.15.19:0/336986830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f10108a20 msgr2=0x7f8f1019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0ffff640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f10108a20 0x7f8f1019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0ffff640 1 -- 172.21.15.19:0/336986830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f10102af0 msgr2=0x7f8f1019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0ffff640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f10102af0 0x7f8f1019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0ffff640 1 -- 172.21.15.19:0/336986830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1019c230 con 0x7f8f1010d380 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0effd640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f10108a20 0x7f8f1019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f10102af0 0x7f8f1019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0ffff640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1010d380 0x7f8f101a6580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800eaa0 tx=0x7f8f0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0cff9640 1 -- 172.21.15.19:0/336986830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800cdb0 con 0x7f8f1010d380 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0cff9640 1 -- 172.21.15.19:0/336986830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f08005020 con 0x7f8f1010d380 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101a6b20 con 0x7f8f1010d380 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f0cff9640 1 -- 172.21.15.19:0/336986830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f080116d0 con 0x7f8f1010d380 2024-04-25T10:58:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.612+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101a7040 con 0x7f8f1010d380 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0cff9640 1 -- 172.21.15.19:0/336986830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8f080051d0 con 0x7f8f1010d380 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 Processor -- start 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 -- start start 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561153824390 0x561153bc5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bc5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bd26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561153b2ad80 con 0x5611538036b0 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561153ae4a20 con 0x561153824390 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561153b26d40 con 0x5611537fb4d0 2024-04-25T10:58:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bd26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bc5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bd26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42598/0 (socket says 172.21.15.19:42598) 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0effd640 1 -- 172.21.15.19:0/3226223543 learned_addr learned my addr 172.21.15.19:0/3226223543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0cff9640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8ef80afce0 0x7f8ef80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0effd640 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561153824390 0x561153bc5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.615+0000 7f8f0ffff640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8ef80afce0 0x7f8ef80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0cff9640 1 -- 172.21.15.19:0/336986830 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8f080df0d0 con 0x7f8f1010d380 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0f7fe640 1 -- 172.21.15.19:0/3226223543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561153824390 msgr2=0x561153bc5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561153824390 0x561153bc5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0f7fe640 1 -- 172.21.15.19:0/3226223543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 msgr2=0x561153bd26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bd26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0f7fe640 1 -- 172.21.15.19:0/3226223543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561153aefc30 con 0x5611537fb4d0 2024-04-25T10:58:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bc5d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f10103a20 tx=0x7f8f0000d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758f72d4633bafc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8f0ffff640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8ef80afce0 0x7f8ef80b21f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8f080dc9a0 tx=0x7f8f08005890 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8ef5ffb640 1 -- 172.21.15.19:0/3226223543 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00017420 con 0x5611537fb4d0 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8ef5ffb640 1 -- 172.21.15.19:0/3226223543 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f00017a70 con 0x5611537fb4d0 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.616+0000 7f8ef5ffb640 1 -- 172.21.15.19:0/3226223543 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0000fa30 con 0x5611537fb4d0 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/3226223543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 msgr2=0x561153bc5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bc5d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f10103a20 tx=0x7f8f0000d340 comp rx=0 tx=0).stop 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/3226223543 shutdown_connections 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bd26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bc5d50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/3226223543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561153824390 0x561153bc5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/3226223543 >> 172.21.15.19:0/3226223543 conn(0x5611538184e0 msgr2=0x561153bc1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/3226223543 shutdown_connections 2024-04-25T10:58:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/3226223543 wait complete. 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.617+0000 7f8f1c9f3740 1 -- start start 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 1 Processor -- start 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bd0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611538036b0 0x561153bd0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bdc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561153bd2d90 con 0x561153824390 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561153ab1dc0 con 0x5611538036b0 2024-04-25T10:58:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561153bd2f10 con 0x5611537fb4d0 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f1c9f3740 10 client.0 _ll_register_callbacks cb 0x561153acd600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bdc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bdc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42600/0 (socket says 172.21.15.19:42600) 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f0effd640 1 -- 172.21.15.19:0/4048186861 learned_addr learned my addr 172.21.15.19:0/4048186861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f0effd640 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bd0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.618+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611538036b0 0x561153bd0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f0effd640 1 -- 172.21.15.19:0/4048186861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 msgr2=0x561153bd0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f0effd640 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bd0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f0effd640 1 -- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611538036b0 msgr2=0x561153bd0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f0effd640 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611538036b0 0x561153bd0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f0effd640 1 -- 172.21.15.19:0/4048186861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115383c3f0 con 0x561153824390 2024-04-25T10:58:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f0effd640 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bdc210 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f04006010 tx=0x7f8f04040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d41ae877178b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8eeeffd640 1 -- 172.21.15.19:0/4048186861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04035040 con 0x561153824390 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8eeeffd640 1 -- 172.21.15.19:0/4048186861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8f04003c30 con 0x561153824390 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f00017a70 con 0x561153824390 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8eeeffd640 1 -- 172.21.15.19:0/4048186861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8f04031910 con 0x561153824390 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f1c9f3740 10 client.16146 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5611539067d0 con 0x561153824390 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.619+0000 7f8eeeffd640 1 -- 172.21.15.19:0/4048186861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f00017a70 con 0x561153824390 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8eeeffd640 1 -- 172.21.15.19:0/4048186861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8f0403a040 con 0x561153824390 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f1c9f3740 10 client.16146 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f1c9f3740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f1c9f3740 20 client.16146 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537e2f60 0x561153be2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f1c9f3740 4 client.16146 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561153b38990 con 0x5611537e2f60 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537e2f60 0x561153be2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8eeeffd640 10 client.16146 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.620+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537e2f60 0x561153be2670 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea47abde65c0c30 server_cookie=683a20375f4b69b3 in_seq=0 out_seq=0 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.621+0000 7f8eeeffd640 1 -- 172.21.15.19:0/4048186861 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561153b38990 con 0x5611537e2f60 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.621+0000 7f8eeeffd640 10 client.16146 handle_command_reply: tid=0 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.621+0000 7f8f1c9f3740 1 client.16146 shutdown 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.621+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 msgr2=0x561153bdc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.621+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bdc210 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f04006010 tx=0x7f8f04040040 comp rx=0 tx=0).stop 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 shutdown_connections 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537e2f60 0x561153be2670 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bdc210 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611538036b0 0x561153bd0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/4048186861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611537fb4d0 0x561153bd0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8eeeffd640 0 client.16146 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 >> 172.21.15.19:0/4048186861 conn(0x5611538184e0 msgr2=0x561153bc7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 shutdown_connections 2024-04-25T10:58:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/4048186861 wait complete. 2024-04-25T10:58:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.622+0000 7f8f1c9f3740 20 client.16146 trim_cache size 0 max 16384 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.635+0000 7f8f1c9f3740 1 Processor -- start 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.635+0000 7f8f1c9f3740 1 -- start start 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bc1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bdbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561153824390 0x561153be7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101ad940 con 0x5611538036b0 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101b01c0 con 0x5611537fb4d0 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101adfb0 con 0x561153824390 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bdbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bdbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42606/0 (socket says 172.21.15.19:42606) 2024-04-25T10:58:19.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 -- 172.21.15.19:0/166484450 learned_addr learned my addr 172.21.15.19:0/166484450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0effd640 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561153824390 0x561153be7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 -- 172.21.15.19:0/166484450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561153824390 msgr2=0x561153be7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561153824390 0x561153be7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0effd640 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bc1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 -- 172.21.15.19:0/166484450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 msgr2=0x561153bc1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bc1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 -- 172.21.15.19:0/166484450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561153bc55a0 con 0x5611538036b0 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bdbe70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d840 tx=0x7f8f0000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a5c55fb7eb8288 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8ef57fa640 1 -- 172.21.15.19:0/166484450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0002a070 con 0x5611538036b0 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8ef57fa640 1 -- 172.21.15.19:0/166484450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f0000de00 con 0x5611538036b0 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.636+0000 7f8ef57fa640 1 -- 172.21.15.19:0/166484450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0000faf0 con 0x5611538036b0 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/166484450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 msgr2=0x561153bdbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bdbe70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d840 tx=0x7f8f0000f520 comp rx=0 tx=0).stop 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/166484450 shutdown_connections 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561153824390 0x561153be7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611538036b0 0x561153bdbe70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/166484450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bc1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/166484450 >> 172.21.15.19:0/166484450 conn(0x5611538184e0 msgr2=0x561153bc17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/166484450 shutdown_connections 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.637+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/166484450 wait complete. 2024-04-25T10:58:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 -- start start 2024-04-25T10:58:19.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 Processor -- start 2024-04-25T10:58:19.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bd3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611538036b0 0x561153bd43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bd4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101ae680 con 0x561153824390 2024-04-25T10:58:19.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101afaf0 con 0x5611537fb4d0 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101aed50 con 0x5611538036b0 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f1c9f3740 10 client.0 _ll_register_callbacks cb 0x7f8f040314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bd4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bd4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42616/0 (socket says 172.21.15.19:42616) 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f0effd640 1 -- 172.21.15.19:0/2233046051 learned_addr learned my addr 172.21.15.19:0/2233046051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611538036b0 0x561153bd43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.638+0000 7f8f0effd640 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bd3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f0effd640 1 -- 172.21.15.19:0/2233046051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611538036b0 msgr2=0x561153bd43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f0effd640 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611538036b0 0x561153bd43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f0effd640 1 -- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 msgr2=0x561153bd3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f0effd640 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bd3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f0effd640 1 -- 172.21.15.19:0/2233046051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561153bcbbf0 con 0x561153824390 2024-04-25T10:58:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f0effd640 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bd4920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f04005f70 tx=0x7f8f04040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca5b98035d5d6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8eeeffd640 1 -- 172.21.15.19:0/2233046051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04058030 con 0x561153824390 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8eeeffd640 1 -- 172.21.15.19:0/2233046051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f8f04031c80 con 0x561153824390 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f0000de00 con 0x561153824390 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8eeeffd640 1 -- 172.21.15.19:0/2233046051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8f0404b400 con 0x561153824390 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f1c9f3740 10 client.16152 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561153c4eee0 con 0x561153824390 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.639+0000 7f8eeeffd640 1 -- 172.21.15.19:0/2233046051 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f0000de00 con 0x561153824390 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8eeeffd640 1 -- 172.21.15.19:0/2233046051 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8f0404bc30 con 0x561153824390 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f1c9f3740 10 client.16152 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f1c9f3740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:19.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f1c9f3740 20 client.16152 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537b83a0 0x561153c51550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f1c9f3740 4 client.16152 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561153b32800 con 0x5611537b83a0 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537b83a0 0x561153c51550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8eeeffd640 10 client.16152 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.640+0000 7f8f0f7fe640 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537b83a0 0x561153c51550 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edeea32e647fbd23 server_cookie=ff2371276329c3c3 in_seq=0 out_seq=0 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8eeeffd640 1 -- 172.21.15.19:0/2233046051 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561153b32800 con 0x5611537b83a0 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8eeeffd640 10 client.16152 handle_command_reply: tid=0 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 client.16152 shutdown 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 msgr2=0x561153bd4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bd4920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f04005f70 tx=0x7f8f04040040 comp rx=0 tx=0).stop 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 shutdown_connections 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5611537b83a0 0x561153c51550 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561153824390 0x561153bd4920 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611538036b0 0x561153bd43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 --2- 172.21.15.19:0/2233046051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611537fb4d0 0x561153bd3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8eeeffd640 0 client.16152 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 >> 172.21.15.19:0/2233046051 conn(0x5611538184e0 msgr2=0x561153bce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 shutdown_connections 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 1 -- 172.21.15.19:0/2233046051 wait complete. 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.641+0000 7f8f1c9f3740 20 client.16152 trim_cache size 0 max 16384 2024-04-25T10:58:19.648 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:58:19.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8ef80afce0 msgr2=0x7f8ef80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8ef80afce0 0x7f8ef80b21f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8f080dc9a0 tx=0x7f8f08005890 comp rx=0 tx=0).stop 2024-04-25T10:58:19.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1010d380 msgr2=0x7f8f101a6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:19.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1010d380 0x7f8f101a6580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800eaa0 tx=0x7f8f0800ef60 comp rx=0 tx=0).stop 2024-04-25T10:58:19.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 shutdown_connections 2024-04-25T10:58:19.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8ef80afce0 0x7f8ef80b21f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1010d380 0x7f8f101a6580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f10108a20 0x7f8f1019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 --2- 172.21.15.19:0/336986830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f10102af0 0x7f8f1019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:19.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 >> 172.21.15.19:0/336986830 conn(0x7f8f10078100 msgr2=0x7f8f101040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:19.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 shutdown_connections 2024-04-25T10:58:19.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:19.642+0000 7f8f15a70640 1 -- 172.21.15.19:0/336986830 wait complete. 2024-04-25T10:58:20.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:20 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2292761219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:58:20.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:20 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2292761219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:58:20.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:20 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2292761219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:58:20.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:58:20.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:58:20.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:58:20.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.784+0000 7fc6fc101640 1 -- 172.21.15.19:0/1887495094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108a20 msgr2=0x7fc6f4102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.784+0000 7fc6fc101640 1 --2- 172.21.15.19:0/1887495094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108a20 0x7fc6f4102620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009900 tx=0x7fc6f002a3f0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.784+0000 7fc6fc101640 1 -- 172.21.15.19:0/1887495094 shutdown_connections 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.784+0000 7fc6fc101640 1 --2- 172.21.15.19:0/1887495094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6f41031d0 0x7fc6f410f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.784+0000 7fc6fc101640 1 --2- 172.21.15.19:0/1887495094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4108ec0 0x7fc6f4102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.784+0000 7fc6fc101640 1 --2- 172.21.15.19:0/1887495094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108a20 0x7fc6f4102620 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.784+0000 7fc6fc101640 1 -- 172.21.15.19:0/1887495094 >> 172.21.15.19:0/1887495094 conn(0x7fc6f4078180 msgr2=0x7fc6f40785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.785+0000 7fc6fc101640 1 -- 172.21.15.19:0/1887495094 shutdown_connections 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.785+0000 7fc6fc101640 1 -- 172.21.15.19:0/1887495094 wait complete. 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.785+0000 7fc6fc101640 1 Processor -- start 2024-04-25T10:58:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.785+0000 7fc6fc101640 1 -- start start 2024-04-25T10:58:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fc101640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6f41031d0 0x7fc6f4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fc101640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4108a20 0x7fc6f4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fc101640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108ec0 0x7fc6f419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fc101640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112c20 con 0x7fc6f4108ec0 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fc101640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112aa0 con 0x7fc6f4108a20 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fc101640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112da0 con 0x7fc6f41031d0 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108ec0 0x7fc6f419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108ec0 0x7fc6f419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42644/0 (socket says 172.21.15.19:42644) 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 -- 172.21.15.19:0/349656664 learned_addr learned my addr 172.21.15.19:0/349656664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6f41031d0 0x7fc6f4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6f969c640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4108a20 0x7fc6f4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 -- 172.21.15.19:0/349656664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6f41031d0 msgr2=0x7fc6f4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6f41031d0 0x7fc6f4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 -- 172.21.15.19:0/349656664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4108a20 msgr2=0x7fc6f4199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4108a20 0x7fc6f4199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 -- 172.21.15.19:0/349656664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f419e420 con 0x7fc6f4108ec0 2024-04-25T10:58:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.786+0000 7fc6fa69e640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108ec0 0x7fc6f419de60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00ead0 tx=0x7fc6ec00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.787+0000 7fc6eaffd640 1 -- 172.21.15.19:0/349656664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00cde0 con 0x7fc6f4108ec0 2024-04-25T10:58:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.787+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f419e710 con 0x7fc6f4108ec0 2024-04-25T10:58:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.787+0000 7fc6eaffd640 1 -- 172.21.15.19:0/349656664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6ec005020 con 0x7fc6f4108ec0 2024-04-25T10:58:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.787+0000 7fc6eaffd640 1 -- 172.21.15.19:0/349656664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0116d0 con 0x7fc6f4108ec0 2024-04-25T10:58:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.787+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f419ec60 con 0x7fc6f4108ec0 2024-04-25T10:58:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.789+0000 7fc702fb0740 1 Processor -- start 2024-04-25T10:58:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.789+0000 7fc6eaffd640 1 -- 172.21.15.19:0/349656664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc6ec011870 con 0x7fc6f4108ec0 2024-04-25T10:58:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc702fb0740 1 -- start start 2024-04-25T10:58:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc702fb0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e5f2328b00 con 0x55e5f2022410 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc702fb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e5f22c4920 con 0x55e5f2001730 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc702fb0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e5f2324ac0 con 0x55e5f1ff9550 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42650/0 (socket says 172.21.15.19:42650) 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 -- 172.21.15.19:0/3286004588 learned_addr learned my addr 172.21.15.19:0/3286004588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6eaffd640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6e00afc60 0x7fc6e00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 -- 172.21.15.19:0/3286004588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 msgr2=0x55e5f23c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 -- 172.21.15.19:0/3286004588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 msgr2=0x55e5f23c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6f969c640 1 -- 172.21.15.19:0/3286004588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f21027e0 con 0x55e5f2022410 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.790+0000 7fc6fa69e640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6e00afc60 0x7fc6e00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.791+0000 7fc6eaffd640 1 -- 172.21.15.19:0/349656664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc6ec0df0b0 con 0x7fc6f4108ec0 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.791+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c4830 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40124a0 tx=0x7fc6e400a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f10871e2fed74e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.791+0000 7fc6dffff640 1 -- 172.21.15.19:0/3286004588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4018030 con 0x55e5f2022410 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.791+0000 7fc6dffff640 1 -- 172.21.15.19:0/3286004588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4005020 con 0x55e5f2022410 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.791+0000 7fc6dffff640 1 -- 172.21.15.19:0/3286004588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4004b20 con 0x55e5f2022410 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 -- 172.21.15.19:0/3286004588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 msgr2=0x55e5f23c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc6fa69e640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6e00afc60 0x7fc6e00b2170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0dc980 tx=0x7fc6ec015000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c4830 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40124a0 tx=0x7fc6e400a5d0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 -- 172.21.15.19:0/3286004588 shutdown_connections 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3286004588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c4830 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 -- 172.21.15.19:0/3286004588 >> 172.21.15.19:0/3286004588 conn(0x55e5f2016560 msgr2=0x55e5f23c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 -- 172.21.15.19:0/3286004588 shutdown_connections 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 -- 172.21.15.19:0/3286004588 wait complete. 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 -- start start 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.792+0000 7fc702fb0740 1 Processor -- start 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2001730 0x55e5f23cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2022410 0x55e5f23db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc702fb0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e5f23d1b80 con 0x55e5f2001730 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc702fb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e5f22c25d0 con 0x55e5f1ff9550 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc702fb0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e5f23d1d00 con 0x55e5f2022410 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc702fb0740 10 client.0 _ll_register_callbacks cb 0x55e5f22b0080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc6f9e9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2001730 0x55e5f23cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc6f9e9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2001730 0x55e5f23cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42664/0 (socket says 172.21.15.19:42664) 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc6f9e9d640 1 -- 172.21.15.19:0/4067428940 learned_addr learned my addr 172.21.15.19:0/4067428940 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc6f969c640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.793+0000 7fc6f969c640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2022410 0x55e5f23db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6f9e9d640 1 -- 172.21.15.19:0/4067428940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2022410 msgr2=0x55e5f23db030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2022410 0x55e5f23db030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6f9e9d640 1 -- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 msgr2=0x55e5f23cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6f9e9d640 1 -- 172.21.15.19:0/4067428940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f2162f70 con 0x55e5f2001730 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6f969c640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2022410 0x55e5f23db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2001730 0x55e5f23cf380 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009900 tx=0x7fc6f00314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d0329f4216aa12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/4067428940 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0035040 con 0x55e5f2001730 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/4067428940 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc6f0004040 con 0x55e5f2001730 2024-04-25T10:58:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e4005020 con 0x55e5f2001730 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/4067428940 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc6f00318f0 con 0x55e5f2001730 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc702fb0740 10 client.16164 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e5f1ffa170 con 0x55e5f2001730 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.794+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/4067428940 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4005020 con 0x55e5f2001730 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/4067428940 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc6f0004910 con 0x55e5f2001730 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc702fb0740 10 client.16164 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc702fb0740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc702fb0740 20 client.16164 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc702fb0740 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fe11f0 0x55e5f23e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc702fb0740 4 client.16164 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f2336710 con 0x55e5f1fe11f0 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc6f969c640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fe11f0 0x55e5f23e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc6dd7fa640 10 client.16164 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.795+0000 7fc6f969c640 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fe11f0 0x55e5f23e1230 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73edb402a5d50f61 server_cookie=1d08dd3b6dc5c522 in_seq=0 out_seq=0 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.796+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/4067428940 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5f2336710 con 0x55e5f1fe11f0 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.796+0000 7fc6dd7fa640 10 client.16164 handle_command_reply: tid=0 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.796+0000 7fc702fb0740 1 client.16164 shutdown 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.796+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2001730 msgr2=0x55e5f23cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.796+0000 7fc702fb0740 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2001730 0x55e5f23cf380 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009900 tx=0x7fc6f00314c0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 shutdown_connections 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fe11f0 0x55e5f23e1230 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2022410 0x55e5f23db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2001730 0x55e5f23cf380 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 --2- 172.21.15.19:0/4067428940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc6dd7fa640 0 client.16164 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 >> 172.21.15.19:0/4067428940 conn(0x55e5f2016560 msgr2=0x55e5f23bd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 shutdown_connections 2024-04-25T10:58:20.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 1 -- 172.21.15.19:0/4067428940 wait complete. 2024-04-25T10:58:20.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.797+0000 7fc702fb0740 20 client.16164 trim_cache size 0 max 16384 2024-04-25T10:58:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 Processor -- start 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 -- start start 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2001730 0x55e5f23c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ab720 con 0x55e5f2022410 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41adfa0 con 0x55e5f1ff9550 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.815+0000 7fc702fb0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41abd90 con 0x55e5f2001730 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42678/0 (socket says 172.21.15.19:42678) 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 -- 172.21.15.19:0/1437224687 learned_addr learned my addr 172.21.15.19:0/1437224687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2001730 0x55e5f23c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 -- 172.21.15.19:0/1437224687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2001730 msgr2=0x55e5f23c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2001730 0x55e5f23c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 -- 172.21.15.19:0/1437224687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 msgr2=0x55e5f23c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 -- 172.21.15.19:0/1437224687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f23b9160 con 0x55e5f2022410 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6f969c640 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c5210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4012df0 tx=0x7fc6e400ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a811a90b9eb4459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6df7fe640 1 -- 172.21.15.19:0/1437224687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4005550 con 0x55e5f2022410 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6df7fe640 1 -- 172.21.15.19:0/1437224687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4031070 con 0x55e5f2022410 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.816+0000 7fc6df7fe640 1 -- 172.21.15.19:0/1437224687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4004920 con 0x55e5f2022410 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 -- 172.21.15.19:0/1437224687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 msgr2=0x55e5f23c5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c5210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4012df0 tx=0x7fc6e400ae50 comp rx=0 tx=0).stop 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 -- 172.21.15.19:0/1437224687 shutdown_connections 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23c5210 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f2001730 0x55e5f23c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 --2- 172.21.15.19:0/1437224687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f1ff9550 0x55e5f23c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 -- 172.21.15.19:0/1437224687 >> 172.21.15.19:0/1437224687 conn(0x55e5f2016560 msgr2=0x55e5f23c45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 -- 172.21.15.19:0/1437224687 shutdown_connections 2024-04-25T10:58:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 -- 172.21.15.19:0/1437224687 wait complete. 2024-04-25T10:58:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 -- start start 2024-04-25T10:58:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.817+0000 7fc702fb0740 1 Processor -- start 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc702fb0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc702fb0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ac460 con 0x55e5f2022410 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc702fb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ad8d0 con 0x55e5f2001730 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc702fb0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41acb30 con 0x55e5f1ff9550 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc702fb0740 10 client.0 _ll_register_callbacks cb 0x7fc6f0032ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42690/0 (socket says 172.21.15.19:42690) 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 -- 172.21.15.19:0/3002297094 learned_addr learned my addr 172.21.15.19:0/3002297094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 -- 172.21.15.19:0/3002297094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 msgr2=0x55e5f23d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 -- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 msgr2=0x55e5f23d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.818+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6f969c640 1 -- 172.21.15.19:0/3002297094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f24485a0 con 0x55e5f2022410 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6f969c640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23d34d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400ade0 tx=0x7fc6e402cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c675823b838a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/3002297094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e402c760 con 0x55e5f2022410 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/3002297094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc6e402c900 con 0x55e5f2022410 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e4031070 con 0x55e5f2022410 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/3002297094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc6e4004830 con 0x55e5f2022410 2024-04-25T10:58:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc702fb0740 10 client.16170 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e5f24496b0 con 0x55e5f2022410 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.819+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/3002297094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4031070 con 0x55e5f2022410 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/3002297094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc6e400f040 con 0x55e5f2022410 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc702fb0740 10 client.16170 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc702fb0740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc702fb0740 20 client.16170 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fb6630 0x55e5f244bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc702fb0740 4 client.16170 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e5f2330580 con 0x55e5f1fb6630 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fb6630 0x55e5f244bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc6dd7fa640 10 client.16170 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.820+0000 7fc6f9e9d640 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fb6630 0x55e5f244bba0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163241a9a2210fea server_cookie=cfc27aa492fc41c in_seq=0 out_seq=0 2024-04-25T10:58:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.821+0000 7fc6dd7fa640 1 -- 172.21.15.19:0/3002297094 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e5f2330580 con 0x55e5f1fb6630 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.821+0000 7fc6dd7fa640 10 client.16170 handle_command_reply: tid=0 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.821+0000 7fc702fb0740 1 client.16170 shutdown 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.821+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 msgr2=0x55e5f23d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.821+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23d34d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400ade0 tx=0x7fc6e402cb40 comp rx=0 tx=0).stop 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 shutdown_connections 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e5f1fb6630 0x55e5f244bba0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5f2022410 0x55e5f23d34d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5f2001730 0x55e5f23d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 --2- 172.21.15.19:0/3002297094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5f1ff9550 0x55e5f23d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6dd7fa640 0 client.16170 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 >> 172.21.15.19:0/3002297094 conn(0x55e5f2016560 msgr2=0x55e5f23c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 shutdown_connections 2024-04-25T10:58:20.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 1 -- 172.21.15.19:0/3002297094 wait complete. 2024-04-25T10:58:20.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc702fb0740 20 client.16170 trim_cache size 0 max 16384 2024-04-25T10:58:20.828 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:58:20.828 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:58:20.828 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "5c2a0a43-d903-4a75-8c74-bb8155501d61", 2024-04-25T10:58:20.828 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:58:20.828 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:58:20.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6e00afc60 msgr2=0x7fc6e00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6e00afc60 0x7fc6e00b2170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0dc980 tx=0x7fc6ec015000 comp rx=0 tx=0).stop 2024-04-25T10:58:20.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108ec0 msgr2=0x7fc6f419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108ec0 0x7fc6f419de60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00ead0 tx=0x7fc6ec00ef90 comp rx=0 tx=0).stop 2024-04-25T10:58:20.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 shutdown_connections 2024-04-25T10:58:20.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6e00afc60 0x7fc6e00b2170 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f4108ec0 0x7fc6f419de60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4108a20 0x7fc6f4199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 --2- 172.21.15.19:0/349656664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6f41031d0 0x7fc6f4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 >> 172.21.15.19:0/349656664 conn(0x7fc6f4078180 msgr2=0x7fc6f4104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 shutdown_connections 2024-04-25T10:58:20.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.822+0000 7fc6fc101640 1 -- 172.21.15.19:0/349656664 wait complete. 2024-04-25T10:58:20.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:58:20.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:58:20.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 -- 172.21.15.19:0/798102511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764104260 msgr2=0x7fe764110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 --2- 172.21.15.19:0/798102511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764104260 0x7fe764110630 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ca0 tx=0x7fe75c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 -- 172.21.15.19:0/798102511 shutdown_connections 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 --2- 172.21.15.19:0/798102511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764104260 0x7fe764110630 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 --2- 172.21.15.19:0/798102511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe764109fb0 0x7fe764103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 --2- 172.21.15.19:0/798102511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764109b10 0x7fe7641036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 -- 172.21.15.19:0/798102511 >> 172.21.15.19:0/798102511 conn(0x7fe7640ff1d0 msgr2=0x7fe764101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 -- 172.21.15.19:0/798102511 shutdown_connections 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 -- 172.21.15.19:0/798102511 wait complete. 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.948+0000 7fe769e4c640 1 Processor -- start 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe769e4c640 1 -- start start 2024-04-25T10:58:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe769e4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764104260 0x7fe76419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe769e4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764109b10 0x7fe76419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe769e4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe764109fb0 0x7fe76419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe769e4c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe764113d00 con 0x7fe764109b10 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe769e4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe764113b80 con 0x7fe764109fb0 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe769e4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe764113e80 con 0x7fe764104260 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764109b10 0x7fe76419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764109b10 0x7fe76419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42720/0 (socket says 172.21.15.19:42720) 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 -- 172.21.15.19:0/3033893675 learned_addr learned my addr 172.21.15.19:0/3033893675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe7637fe640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764104260 0x7fe76419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe763fff640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe764109fb0 0x7fe76419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 -- 172.21.15.19:0/3033893675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764104260 msgr2=0x7fe76419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764104260 0x7fe76419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 -- 172.21.15.19:0/3033893675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe764109fb0 msgr2=0x7fe76419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe764109fb0 0x7fe76419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 -- 172.21.15.19:0/3033893675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76419f560 con 0x7fe764109b10 2024-04-25T10:58:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.949+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764109b10 0x7fe76419ac60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe75800d740 tx=0x7fe75800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.950+0000 7fe760ff9640 1 -- 172.21.15.19:0/3033893675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758017070 con 0x7fe764109b10 2024-04-25T10:58:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.950+0000 7fe760ff9640 1 -- 172.21.15.19:0/3033893675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75800de50 con 0x7fe764109b10 2024-04-25T10:58:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.950+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76419f850 con 0x7fe764109b10 2024-04-25T10:58:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.950+0000 7fe760ff9640 1 -- 172.21.15.19:0/3033893675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758003c20 con 0x7fe764109b10 2024-04-25T10:58:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.950+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641a71d0 con 0x7fe764109b10 2024-04-25T10:58:20.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.952+0000 7fe770ddc740 1 Processor -- start 2024-04-25T10:58:20.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.952+0000 7fe760ff9640 1 -- 172.21.15.19:0/3033893675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe758003dc0 con 0x7fe764109b10 2024-04-25T10:58:20.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.952+0000 7fe770ddc740 1 -- start start 2024-04-25T10:58:20.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a0ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a0bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b064a6b0 0x5646b0a188f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe770ddc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646b0971d80 con 0x5646b064a6b0 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe770ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646b092b8b0 con 0x5646b06424d0 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe770ddc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646b096dd40 con 0x5646b066b390 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b064a6b0 0x5646b0a188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b064a6b0 0x5646b0a188f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42722/0 (socket says 172.21.15.19:42722) 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 -- 172.21.15.19:0/3985899024 learned_addr learned my addr 172.21.15.19:0/3985899024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe7637fe640 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a0bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe763fff640 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a0ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe760ff9640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe74c0afce0 0x7fe74c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 -- 172.21.15.19:0/3985899024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 msgr2=0x5646b0a0ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a0ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 -- 172.21.15.19:0/3985899024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 msgr2=0x5646b0a0bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a0bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe762ffd640 1 -- 172.21.15.19:0/3985899024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b08fca40 con 0x5646b064a6b0 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.953+0000 7fe763fff640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe74c0afce0 0x7fe74c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.954+0000 7fe760ff9640 1 -- 172.21.15.19:0/3033893675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe7580dea20 con 0x7fe764109b10 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.954+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b064a6b0 0x5646b0a188f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe7580dc110 tx=0x7fe75806d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8beb026fb46b18f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.954+0000 7fe749ffb640 1 -- 172.21.15.19:0/3985899024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580a69c0 con 0x5646b064a6b0 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.954+0000 7fe763fff640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe74c0afce0 0x7fe74c0b21f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005cc0 tx=0x7fe75c0095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.954+0000 7fe749ffb640 1 -- 172.21.15.19:0/3985899024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7580a6700 con 0x5646b064a6b0 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.954+0000 7fe749ffb640 1 -- 172.21.15.19:0/3985899024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580a6360 con 0x5646b064a6b0 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 -- 172.21.15.19:0/3985899024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b064a6b0 msgr2=0x5646b0a188f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b064a6b0 0x5646b0a188f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe7580dc110 tx=0x7fe75806d010 comp rx=0 tx=0).stop 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 -- 172.21.15.19:0/3985899024 shutdown_connections 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b064a6b0 0x5646b0a188f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a0bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3985899024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a0ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 -- 172.21.15.19:0/3985899024 >> 172.21.15.19:0/3985899024 conn(0x5646b065f4e0 msgr2=0x5646b0a08210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 -- 172.21.15.19:0/3985899024 shutdown_connections 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 -- 172.21.15.19:0/3985899024 wait complete. 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 -- start start 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.955+0000 7fe770ddc740 1 Processor -- start 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b064a6b0 0x5646b0a16b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a1c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe770ddc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646b0a18f20 con 0x5646b06424d0 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe770ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646b08f8d10 con 0x5646b064a6b0 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe770ddc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646b0a190a0 con 0x5646b066b390 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe770ddc740 10 client.0 _ll_register_callbacks cb 0x5646b0925210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42736/0 (socket says 172.21.15.19:42736) 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 -- 172.21.15.19:0/4245899930 learned_addr learned my addr 172.21.15.19:0/4245899930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe763fff640 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a1c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe762ffd640 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b064a6b0 0x5646b0a16b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 -- 172.21.15.19:0/4245899930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 msgr2=0x5646b0a1c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a1c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 -- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b064a6b0 msgr2=0x5646b0a16b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b064a6b0 0x5646b0a16b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.956+0000 7fe7637fe640 1 -- 172.21.15.19:0/4245899930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b074d7d0 con 0x5646b06424d0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe7637fe640 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a165e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe754012700 tx=0x7fe754012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4267caf2517cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe742ffd640 1 -- 172.21.15.19:0/4245899930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754004260 con 0x5646b06424d0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe742ffd640 1 -- 172.21.15.19:0/4245899930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe754004400 con 0x5646b06424d0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7580a6700 con 0x5646b06424d0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe742ffd640 1 -- 172.21.15.19:0/4245899930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe7540119a0 con 0x5646b06424d0 2024-04-25T10:58:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 10 client.16182 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5646b06833f0 con 0x5646b06424d0 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe742ffd640 1 -- 172.21.15.19:0/4245899930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7580a6700 con 0x5646b06424d0 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe742ffd640 1 -- 172.21.15.19:0/4245899930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe75400f440 con 0x5646b06424d0 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 10 client.16182 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 20 client.16182 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b0629f60 0x5646b0a286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 4 client.16182 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.957+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646b097f990 con 0x5646b0629f60 2024-04-25T10:58:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.958+0000 7fe762ffd640 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b0629f60 0x5646b0a286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.958+0000 7fe742ffd640 10 client.16182 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.958+0000 7fe762ffd640 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b0629f60 0x5646b0a286d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c4b05e5d1cb357 server_cookie=c1156d49c7218a4a in_seq=0 out_seq=0 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe742ffd640 1 -- 172.21.15.19:0/4245899930 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646b097f990 con 0x5646b0629f60 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe742ffd640 10 client.16182 handle_command_reply: tid=0 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 client.16182 shutdown 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 msgr2=0x5646b0a165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a165e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe754012700 tx=0x7fe754012bc0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 shutdown_connections 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b0629f60 0x5646b0a286d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b066b390 0x5646b0a1c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b064a6b0 0x5646b0a16b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 --2- 172.21.15.19:0/4245899930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a165e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe742ffd640 0 client.16182 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 >> 172.21.15.19:0/4245899930 conn(0x5646b065f4e0 msgr2=0x5646b0a0d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 shutdown_connections 2024-04-25T10:58:20.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 1 -- 172.21.15.19:0/4245899930 wait complete. 2024-04-25T10:58:20.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.959+0000 7fe770ddc740 20 client.16182 trim_cache size 0 max 16384 2024-04-25T10:58:20.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 Processor -- start 2024-04-25T10:58:20.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 -- start start 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b065f4e0 0x5646b0a156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641ad980 con 0x5646b06424d0 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b0200 con 0x5646b065f4e0 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe770ddc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641adff0 con 0x5646b064a6b0 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a0b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42738/0 (socket says 172.21.15.19:42738) 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 -- 172.21.15.19:0/3324852638 learned_addr learned my addr 172.21.15.19:0/3324852638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe7637fe640 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b065f4e0 0x5646b0a156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe763fff640 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 -- 172.21.15.19:0/3324852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 msgr2=0x5646b0a1c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 -- 172.21.15.19:0/3324852638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b065f4e0 msgr2=0x5646b0a156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b065f4e0 0x5646b0a156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe762ffd640 1 -- 172.21.15.19:0/3324852638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b06430f0 con 0x5646b06424d0 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.978+0000 7fe763fff640 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.979+0000 7fe762ffd640 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a0b900 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe7580af520 tx=0x7fe7580e24c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318aef2c6a8d9e7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.979+0000 7fe7497fa640 1 -- 172.21.15.19:0/3324852638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758020040 con 0x5646b06424d0 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.979+0000 7fe7497fa640 1 -- 172.21.15.19:0/3324852638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7580e7040 con 0x5646b06424d0 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.979+0000 7fe7497fa640 1 -- 172.21.15.19:0/3324852638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580a6540 con 0x5646b06424d0 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 -- 172.21.15.19:0/3324852638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 msgr2=0x5646b0a0b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a0b900 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe7580af520 tx=0x7fe7580e24c0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 -- 172.21.15.19:0/3324852638 shutdown_connections 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b06424d0 0x5646b0a0b900 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 --2- 172.21.15.19:0/3324852638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b065f4e0 0x5646b0a156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 -- 172.21.15.19:0/3324852638 >> 172.21.15.19:0/3324852638 conn(0x5646b0a0c0d0 msgr2=0x5646b0a0dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 -- 172.21.15.19:0/3324852638 shutdown_connections 2024-04-25T10:58:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 -- 172.21.15.19:0/3324852638 wait complete. 2024-04-25T10:58:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 -- start start 2024-04-25T10:58:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.980+0000 7fe770ddc740 1 Processor -- start 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a1db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b065f4e0 0x5646b0a8ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641ae6c0 con 0x5646b065f4e0 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641afb30 con 0x5646b06424d0 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641aed90 con 0x5646b064a6b0 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 10 client.0 _ll_register_callbacks cb 0x7fe7540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b065f4e0 0x5646b0a8ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b065f4e0 0x5646b0a8ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42748/0 (socket says 172.21.15.19:42748) 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 -- 172.21.15.19:0/2739728568 learned_addr learned my addr 172.21.15.19:0/2739728568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe7637fe640 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a1db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 -- 172.21.15.19:0/2739728568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 msgr2=0x5646b0a1e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe762ffd640 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 -- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 msgr2=0x5646b0a1db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a1db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 -- 172.21.15.19:0/2739728568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b09324b0 con 0x5646b065f4e0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe763fff640 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b065f4e0 0x5646b0a8ea30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ca0 tx=0x7fe75c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63950c760ec76c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe742ffd640 1 -- 172.21.15.19:0/2739728568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c058060 con 0x5646b065f4e0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe742ffd640 1 -- 172.21.15.19:0/2739728568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe75c004510 con 0x5646b065f4e0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.981+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7580e7040 con 0x5646b065f4e0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe742ffd640 1 -- 172.21.15.19:0/2739728568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe75c004710 con 0x5646b065f4e0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 10 client.16188 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5646b0a8ff80 con 0x5646b065f4e0 2024-04-25T10:58:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe742ffd640 1 -- 172.21.15.19:0/2739728568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7580e7040 con 0x5646b065f4e0 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe742ffd640 1 -- 172.21.15.19:0/2739728568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe75c052030 con 0x5646b065f4e0 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 10 client.16188 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 20 client.16188 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b066b390 0x5646b0a92770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 4 client.16188 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646b0979800 con 0x5646b066b390 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.982+0000 7fe7637fe640 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b066b390 0x5646b0a92770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe742ffd640 10 client.16188 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe7637fe640 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b066b390 0x5646b0a92770 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1479049ad856fc75 server_cookie=359d08e35af1b68e in_seq=0 out_seq=0 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe742ffd640 1 -- 172.21.15.19:0/2739728568 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5646b0979800 con 0x5646b066b390 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe742ffd640 10 client.16188 handle_command_reply: tid=0 2024-04-25T10:58:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 client.16188 shutdown 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b065f4e0 msgr2=0x5646b0a8ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b065f4e0 0x5646b0a8ea30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ca0 tx=0x7fe75c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 shutdown_connections 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5646b066b390 0x5646b0a92770 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b065f4e0 0x5646b0a8ea30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b064a6b0 0x5646b0a1e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe742ffd640 0 client.16188 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 --2- 172.21.15.19:0/2739728568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646b06424d0 0x5646b0a1db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 >> 172.21.15.19:0/2739728568 conn(0x5646b0a0c0d0 msgr2=0x5646b0a179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 shutdown_connections 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 1 -- 172.21.15.19:0/2739728568 wait complete. 2024-04-25T10:58:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.983+0000 7fe770ddc740 20 client.16188 trim_cache size 0 max 16384 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: "5c2a0a43-d903-4a75-8c74-bb8155501d61": { 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "5c2a0a43-d903-4a75-8c74-bb8155501d61", 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:58:20.991 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:58:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe74c0afce0 msgr2=0x7fe74c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe74c0afce0 0x7fe74c0b21f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005cc0 tx=0x7fe75c0095c0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764109b10 msgr2=0x7fe76419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764109b10 0x7fe76419ac60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe75800d740 tx=0x7fe75800dc00 comp rx=0 tx=0).stop 2024-04-25T10:58:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 shutdown_connections 2024-04-25T10:58:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe74c0afce0 0x7fe74c0b21f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe764109fb0 0x7fe76419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764109b10 0x7fe76419ac60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 --2- 172.21.15.19:0/3033893675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764104260 0x7fe76419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 >> 172.21.15.19:0/3033893675 conn(0x7fe7640ff1d0 msgr2=0x7fe7641065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 shutdown_connections 2024-04-25T10:58:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:20.984+0000 7fe769e4c640 1 -- 172.21.15.19:0/3033893675 wait complete. 2024-04-25T10:58:20.996 INFO:tasks.cephfs.filesystem:scrub status for tag:5c2a0a43-d903-4a75-8c74-bb8155501d61 - {'path': '/', 'tag': '5c2a0a43-d903-4a75-8c74-bb8155501d61', 'options': 'recursive,force'} 2024-04-25T10:58:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:21 smithi019 ceph-mon[29209]: pgmap v118: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:58:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:21 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:58:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:21 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:21 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:58:21.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:21 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:21 smithi169 ceph-mon[33049]: pgmap v118: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:58:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:21 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:58:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:21 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:21 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:58:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:21 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:21 smithi112 ceph-mon[32799]: pgmap v118: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:58:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:21 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:58:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:21 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:21 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:58:21.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:21 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:22.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:22 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:58:22.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:22 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:58:22.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:22 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:58:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:23 smithi169 ceph-mon[33049]: pgmap v119: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T10:58:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:23 smithi112 ceph-mon[32799]: pgmap v119: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T10:58:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:23 smithi019 ceph-mon[29209]: pgmap v119: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T10:58:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:25 smithi169 ceph-mon[33049]: pgmap v120: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:25 smithi112 ceph-mon[32799]: pgmap v120: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:25 smithi019 ceph-mon[29209]: pgmap v120: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:27 smithi169 ceph-mon[33049]: pgmap v121: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:27 smithi112 ceph-mon[32799]: pgmap v121: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:27 smithi019 ceph-mon[29209]: pgmap v121: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:29.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:29 smithi169 ceph-mon[33049]: pgmap v122: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:29.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:29 smithi112 ceph-mon[32799]: pgmap v122: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:29.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:29 smithi019 ceph-mon[29209]: pgmap v122: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:31.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:31 smithi169 ceph-mon[33049]: pgmap v123: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:31.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:31 smithi112 ceph-mon[32799]: pgmap v123: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:31.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:31 smithi019 ceph-mon[29209]: pgmap v123: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:33.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:33 smithi169 ceph-mon[33049]: pgmap v124: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:33.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:33 smithi112 ceph-mon[32799]: pgmap v124: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:33.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:33 smithi019 ceph-mon[29209]: pgmap v124: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:35.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:35 smithi169 ceph-mon[33049]: pgmap v125: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:35.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:35 smithi112 ceph-mon[32799]: pgmap v125: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:35.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:35 smithi019 ceph-mon[29209]: pgmap v125: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:37 smithi169 ceph-mon[33049]: pgmap v126: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:37.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:37 smithi112 ceph-mon[32799]: pgmap v126: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:37.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:37 smithi019 ceph-mon[29209]: pgmap v126: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:39.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:39 smithi112 ceph-mon[32799]: pgmap v127: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:39.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:39 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:58:39.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:39 smithi019 ceph-mon[29209]: pgmap v127: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:39 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:58:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:39 smithi169 ceph-mon[33049]: pgmap v127: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:39 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:58:40.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:40 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:58:40.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:40 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:58:40.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:40 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:58:40.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:40 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:58:40.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:40 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:58:40.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:40 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:58:40.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:40 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:58:40.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:40 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:58:40.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:40 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:58:41.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:41 smithi112 ceph-mon[32799]: pgmap v128: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:41.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:41 smithi019 ceph-mon[29209]: pgmap v128: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:41.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:41 smithi169 ceph-mon[33049]: pgmap v128: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:43 smithi112 ceph-mon[32799]: pgmap v129: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:43 smithi019 ceph-mon[29209]: pgmap v129: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:43 smithi169 ceph-mon[33049]: pgmap v129: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:45.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:45 smithi112 ceph-mon[32799]: pgmap v130: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:45.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:45 smithi019 ceph-mon[29209]: pgmap v130: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:45.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:45 smithi169 ceph-mon[33049]: pgmap v130: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:47.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:47 smithi112 ceph-mon[32799]: pgmap v131: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:47.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:47 smithi019 ceph-mon[29209]: pgmap v131: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:47.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:47 smithi169 ceph-mon[33049]: pgmap v131: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:49.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:49 smithi112 ceph-mon[32799]: pgmap v132: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:49.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:49 smithi019 ceph-mon[29209]: pgmap v132: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:49.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:49 smithi169 ceph-mon[33049]: pgmap v132: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:50.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:58:50.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:58:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 -- 172.21.15.19:0/2236649857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790108a20 msgr2=0x7f47901025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 --2- 172.21.15.19:0/2236649857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790108a20 0x7f47901025b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4780009900 tx=0x7f478002a410 comp rx=0 tx=0).stop 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 -- 172.21.15.19:0/2236649857 shutdown_connections 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 --2- 172.21.15.19:0/2236649857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479010d380 0x7f479010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 --2- 172.21.15.19:0/2236649857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4790102af0 0x7f4790102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 --2- 172.21.15.19:0/2236649857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790108a20 0x7f47901025b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 -- 172.21.15.19:0/2236649857 >> 172.21.15.19:0/2236649857 conn(0x7f4790078100 msgr2=0x7f4790078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 -- 172.21.15.19:0/2236649857 shutdown_connections 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.126+0000 7f4796b72640 1 -- 172.21.15.19:0/2236649857 wait complete. 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 Processor -- start 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 -- start start 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790102af0 0x7f47900ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4790108a20 0x7f47901001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479010d380 0x7f4790100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4790112be0 con 0x7f4790102af0 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4790112a60 con 0x7f4790108a20 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f4796b72640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4790112d60 con 0x7f479010d380 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f479490e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790102af0 0x7f47900ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f479490e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790102af0 0x7f47900ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49260/0 (socket says 172.21.15.19:49260) 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f479490e640 1 -- 172.21.15.19:0/4249884079 learned_addr learned my addr 172.21.15.19:0/4249884079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.127+0000 7f479510f640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479010d380 0x7f4790100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f478ffff640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4790108a20 0x7f47901001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f479490e640 1 -- 172.21.15.19:0/4249884079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479010d380 msgr2=0x7f4790100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f479490e640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479010d380 0x7f4790100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f479490e640 1 -- 172.21.15.19:0/4249884079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4790108a20 msgr2=0x7f47901001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f479490e640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4790108a20 0x7f47901001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f479490e640 1 -- 172.21.15.19:0/4249884079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47901a8af0 con 0x7f4790102af0 2024-04-25T10:58:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f479490e640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790102af0 0x7f47900ffc80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4780032fd0 tx=0x7f4780031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f478dffb640 1 -- 172.21.15.19:0/4249884079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478003f030 con 0x7f4790102af0 2024-04-25T10:58:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f478dffb640 1 -- 172.21.15.19:0/4249884079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4780004490 con 0x7f4790102af0 2024-04-25T10:58:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47901a8d80 con 0x7f4790102af0 2024-04-25T10:58:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f478dffb640 1 -- 172.21.15.19:0/4249884079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780035070 con 0x7f4790102af0 2024-04-25T10:58:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.128+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47901a91c0 con 0x7f4790102af0 2024-04-25T10:58:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.131+0000 7f479da25740 1 Processor -- start 2024-04-25T10:58:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.131+0000 7f478dffb640 1 -- 172.21.15.19:0/4249884079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4780042030 con 0x7f4790102af0 2024-04-25T10:58:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.131+0000 7f479da25740 1 -- start start 2024-04-25T10:58:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.131+0000 7f479da25740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e6400f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.131+0000 7f479da25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd2b4d0 0x55e6400f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.131+0000 7f479da25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f479da25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e64005ad80 con 0x55e63fd336b0 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f479da25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e6400148b0 con 0x55e63fd2b4d0 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f479da25740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e640056d40 con 0x55e63fd54390 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49262/0 (socket says 172.21.15.19:49262) 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 -- 172.21.15.19:0/58514933 learned_addr learned my addr 172.21.15.19:0/58514933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f479490e640 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd2b4d0 0x55e6400f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f479510f640 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e6400f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478dffb640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f47780afc60 0x7f47780b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 -- 172.21.15.19:0/58514933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 msgr2=0x55e6400f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e6400f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 -- 172.21.15.19:0/58514933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd2b4d0 msgr2=0x55e6400f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd2b4d0 0x55e6400f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 -- 172.21.15.19:0/58514933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e63ffe5a40 con 0x55e63fd336b0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f479490e640 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd2b4d0 0x55e6400f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f479510f640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f47780afc60 0x7f47780b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478dffb640 1 -- 172.21.15.19:0/4249884079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f47800ff3d0 con 0x7f4790102af0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f478ffff640 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401018f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4784009bb0 tx=0x7f47840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6f3e447908e46c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.132+0000 7f4776ffd640 1 -- 172.21.15.19:0/58514933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784005020 con 0x55e63fd336b0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f4776ffd640 1 -- 172.21.15.19:0/58514933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47840047a0 con 0x55e63fd336b0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f4776ffd640 1 -- 172.21.15.19:0/58514933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840126b0 con 0x55e63fd336b0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479da25740 1 -- 172.21.15.19:0/58514933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 msgr2=0x55e6401018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479510f640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f47780afc60 0x7f47780b2170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55e6400f1bd0 tx=0x7f4788014040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479da25740 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401018f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4784009bb0 tx=0x7f47840117b0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479da25740 1 -- 172.21.15.19:0/58514933 shutdown_connections 2024-04-25T10:58:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479da25740 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401018f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479da25740 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd2b4d0 0x55e6400f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479da25740 1 --2- 172.21.15.19:0/58514933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e6400f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.133+0000 7f479da25740 1 -- 172.21.15.19:0/58514933 >> 172.21.15.19:0/58514933 conn(0x55e63fd484e0 msgr2=0x55e6400f1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.134+0000 7f479da25740 1 -- 172.21.15.19:0/58514933 shutdown_connections 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.134+0000 7f479da25740 1 -- 172.21.15.19:0/58514933 wait complete. 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.134+0000 7f479da25740 1 -- start start 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.134+0000 7f479da25740 1 Processor -- start 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.134+0000 7f479da25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd2b4d0 0x55e6400ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479da25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd336b0 0x55e6400ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479da25740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e64010b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479da25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e640101fe0 con 0x55e63fd2b4d0 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479da25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e63ffe1d10 con 0x55e63fd336b0 2024-04-25T10:58:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479da25740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e640102160 con 0x55e63fd54390 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479da25740 10 client.0 _ll_register_callbacks cb 0x55e64000e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd2b4d0 0x55e6400ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd2b4d0 0x55e6400ff580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49264/0 (socket says 172.21.15.19:49264) 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 -- 172.21.15.19:0/1595638023 learned_addr learned my addr 172.21.15.19:0/1595638023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f478ffff640 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd336b0 0x55e6400ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479510f640 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e64010b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 -- 172.21.15.19:0/1595638023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 msgr2=0x55e64010b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e64010b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 -- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd336b0 msgr2=0x55e6400ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd336b0 0x55e6400ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 -- 172.21.15.19:0/1595638023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63fe367d0 con 0x55e63fd2b4d0 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479490e640 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd2b4d0 0x55e6400ff580 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4780031440 tx=0x7f478003b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfe2c8c5c4785c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f476ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f476ffff640 1 -- 172.21.15.19:0/1595638023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780049070 con 0x55e63fd2b4d0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f476ffff640 1 -- 172.21.15.19:0/1595638023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f478003ba30 con 0x55e63fd2b4d0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.135+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47840047a0 con 0x55e63fd2b4d0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f476ffff640 1 -- 172.21.15.19:0/1595638023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f47800c7370 con 0x55e63fd2b4d0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 10 client.16200 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e63fd6c3f0 con 0x55e63fd2b4d0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f476ffff640 1 -- 172.21.15.19:0/1595638023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47840047a0 con 0x55e63fd2b4d0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f476ffff640 1 -- 172.21.15.19:0/1595638023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f47800c7800 con 0x55e63fd2b4d0 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 10 client.16200 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 20 client.16200 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fd12f60 0x55e640111520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 4 client.16200 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e640068990 con 0x55e63fd12f60 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.136+0000 7f478ffff640 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fd12f60 0x55e640111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.137+0000 7f476ffff640 10 client.16200 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.137+0000 7f478ffff640 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fd12f60 0x55e640111520 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4765a38826112d25 server_cookie=27f045437ee63c47 in_seq=0 out_seq=0 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.137+0000 7f476ffff640 1 -- 172.21.15.19:0/1595638023 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e640068990 con 0x55e63fd12f60 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.137+0000 7f476ffff640 10 client.16200 handle_command_reply: tid=0 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 client.16200 shutdown 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd2b4d0 msgr2=0x55e6400ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd2b4d0 0x55e6400ff580 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4780031440 tx=0x7f478003b560 comp rx=0 tx=0).stop 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 shutdown_connections 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fd12f60 0x55e640111520 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd54390 0x55e64010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f476ffff640 0 client.16200 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd336b0 0x55e6400ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 --2- 172.21.15.19:0/1595638023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd2b4d0 0x55e6400ff580 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 >> 172.21.15.19:0/1595638023 conn(0x55e63fd484e0 msgr2=0x55e6400f6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 shutdown_connections 2024-04-25T10:58:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 1 -- 172.21.15.19:0/1595638023 wait complete. 2024-04-25T10:58:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.138+0000 7f479da25740 20 client.16200 trim_cache size 0 max 16384 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.156+0000 7f479da25740 1 Processor -- start 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.156+0000 7f479da25740 1 -- start start 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f479da25740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 0x55e6400fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f479da25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f479da25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 0x55e640107430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f479da25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47901afb20 con 0x55e63fd336b0 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f479da25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b23a0 con 0x55e63fd54390 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f479da25740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b0190 con 0x55e63fd2b4d0 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49276/0 (socket says 172.21.15.19:49276) 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 -- 172.21.15.19:0/1974627577 learned_addr learned my addr 172.21.15.19:0/1974627577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 -- 172.21.15.19:0/1974627577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 msgr2=0x55e6400fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 --2- 172.21.15.19:0/1974627577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 0x55e6400fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 -- 172.21.15.19:0/1974627577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 msgr2=0x55e640107430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 --2- 172.21.15.19:0/1974627577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 0x55e640107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 -- 172.21.15.19:0/1974627577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e63fd2c0f0 con 0x55e63fd336b0 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f478ffff640 1 --2- 172.21.15.19:0/1974627577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401011c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4784000c00 tx=0x7f47840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11895661d8e771f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f47767fc640 1 -- 172.21.15.19:0/1974627577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478401f040 con 0x55e63fd336b0 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f47767fc640 1 -- 172.21.15.19:0/1974627577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47840054d0 con 0x55e63fd336b0 2024-04-25T10:58:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.157+0000 7f47767fc640 1 -- 172.21.15.19:0/1974627577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784004b20 con 0x55e63fd336b0 2024-04-25T10:58:51.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 -- 172.21.15.19:0/1974627577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 msgr2=0x55e6401011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 --2- 172.21.15.19:0/1974627577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401011c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4784000c00 tx=0x7f47840117b0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 -- 172.21.15.19:0/1974627577 shutdown_connections 2024-04-25T10:58:51.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 --2- 172.21.15.19:0/1974627577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 0x55e640107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 --2- 172.21.15.19:0/1974627577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e6401011c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 --2- 172.21.15.19:0/1974627577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 0x55e6400fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 -- 172.21.15.19:0/1974627577 >> 172.21.15.19:0/1974627577 conn(0x55e63fd484e0 msgr2=0x55e640100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 -- 172.21.15.19:0/1974627577 shutdown_connections 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 -- 172.21.15.19:0/1974627577 wait complete. 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 -- start start 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.158+0000 7f479da25740 1 Processor -- start 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479da25740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 0x55e640102a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479da25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e640102f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479da25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 0x55e6401718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479da25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b0860 con 0x55e63fd336b0 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479da25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b1cd0 con 0x55e63fd54390 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479da25740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47901b0f30 con 0x55e63fd2b4d0 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479da25740 10 client.0 _ll_register_callbacks cb 0x7f4780005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e640102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e640102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49290/0 (socket says 172.21.15.19:49290) 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 -- 172.21.15.19:0/1397698 learned_addr learned my addr 172.21.15.19:0/1397698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479490e640 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 0x55e640102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f479510f640 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 0x55e6401718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 -- 172.21.15.19:0/1397698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 msgr2=0x55e640102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 0x55e640102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 -- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 msgr2=0x55e6401718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 0x55e6401718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.159+0000 7f478ffff640 1 -- 172.21.15.19:0/1397698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64001b4b0 con 0x55e63fd336b0 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f478ffff640 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e640102f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4784009940 tx=0x7f4784005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7701593f97110d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f476ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f476ffff640 1 -- 172.21.15.19:0/1397698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840052d0 con 0x55e63fd336b0 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f476ffff640 1 -- 172.21.15.19:0/1397698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f478402d2f0 con 0x55e63fd336b0 2024-04-25T10:58:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47840054d0 con 0x55e63fd336b0 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f476ffff640 1 -- 172.21.15.19:0/1397698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f478402d4f0 con 0x55e63fd336b0 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f479da25740 10 client.16206 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e640178f40 con 0x55e63fd336b0 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f476ffff640 1 -- 172.21.15.19:0/1397698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47840054d0 con 0x55e63fd336b0 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f476ffff640 1 -- 172.21.15.19:0/1397698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f478402dce0 con 0x55e63fd336b0 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f479da25740 10 client.16206 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f479da25740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f479da25740 20 client.16206 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.160+0000 7f479da25740 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fce83a0 0x55e64017b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f479da25740 4 client.16206 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e640062800 con 0x55e63fce83a0 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f479490e640 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fce83a0 0x55e64017b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f476ffff640 10 client.16206 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f479490e640 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fce83a0 0x55e64017b420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6001c8efaa584e74 server_cookie=92eacc4d9e6d1925 in_seq=0 out_seq=0 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f476ffff640 1 -- 172.21.15.19:0/1397698 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e640062800 con 0x55e63fce83a0 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f476ffff640 10 client.16206 handle_command_reply: tid=0 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.161+0000 7f479da25740 1 client.16206 shutdown 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 msgr2=0x55e640102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e640102f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4784009940 tx=0x7f4784005020 comp rx=0 tx=0).stop 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 shutdown_connections 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55e63fce83a0 0x55e64017b420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e63fd54390 0x55e6401718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63fd336b0 0x55e640102f60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 --2- 172.21.15.19:0/1397698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e63fd2b4d0 0x55e640102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 >> 172.21.15.19:0/1397698 conn(0x55e63fd484e0 msgr2=0x55e6400fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f476ffff640 0 client.16206 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 shutdown_connections 2024-04-25T10:58:51.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 1 -- 172.21.15.19:0/1397698 wait complete. 2024-04-25T10:58:51.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f479da25740 20 client.16206 trim_cache size 0 max 16384 2024-04-25T10:58:51.169 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:58:51.169 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:58:51.169 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:58:51.169 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:58:51.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f47780afc60 msgr2=0x7f47780b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f47780afc60 0x7f47780b2170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55e6400f1bd0 tx=0x7f4788014040 comp rx=0 tx=0).stop 2024-04-25T10:58:51.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790102af0 msgr2=0x7f47900ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790102af0 0x7f47900ffc80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4780032fd0 tx=0x7f4780031a20 comp rx=0 tx=0).stop 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 shutdown_connections 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f47780afc60 0x7f47780b2170 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479010d380 0x7f4790100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4790108a20 0x7f47901001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 --2- 172.21.15.19:0/4249884079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4790102af0 0x7f47900ffc80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.162+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 >> 172.21.15.19:0/4249884079 conn(0x7f4790078100 msgr2=0x7f4790069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.163+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 shutdown_connections 2024-04-25T10:58:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.163+0000 7f4796b72640 1 -- 172.21.15.19:0/4249884079 wait complete. 2024-04-25T10:58:51.174 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:58:51.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:58:51.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 -- 172.21.15.19:0/3266431955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730107450 msgr2=0x7fe73010ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 --2- 172.21.15.19:0/3266431955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730107450 0x7fe73010ebf0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe728009900 tx=0x7fe72802a410 comp rx=0 tx=0).stop 2024-04-25T10:58:51.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 -- 172.21.15.19:0/3266431955 shutdown_connections 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 --2- 172.21.15.19:0/3266431955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730107450 0x7fe73010ebf0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 --2- 172.21.15.19:0/3266431955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe730106990 0x7fe730106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 --2- 172.21.15.19:0/3266431955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe730102400 0x7fe730102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 -- 172.21.15.19:0/3266431955 >> 172.21.15.19:0/3266431955 conn(0x7fe730077890 msgr2=0x7fe730077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 -- 172.21.15.19:0/3266431955 shutdown_connections 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.287+0000 7fe73875e640 1 -- 172.21.15.19:0/3266431955 wait complete. 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 Processor -- start 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 -- start start 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe730102400 0x7fe730199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106990 0x7fe730199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe730107450 0x7fe73019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe730112bf0 con 0x7fe730106990 2024-04-25T10:58:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe730112a70 con 0x7fe730107450 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe73875e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe730112d70 con 0x7fe730102400 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe735cf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106990 0x7fe730199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe735cf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106990 0x7fe730199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49304/0 (socket says 172.21.15.19:49304) 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.288+0000 7fe735cf9640 1 -- 172.21.15.19:0/2428396818 learned_addr learned my addr 172.21.15.19:0/2428396818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe736cfb640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe730107450 0x7fe73019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe7364fa640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe730102400 0x7fe730199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe735cf9640 1 -- 172.21.15.19:0/2428396818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe730102400 msgr2=0x7fe730199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe735cf9640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe730102400 0x7fe730199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe735cf9640 1 -- 172.21.15.19:0/2428396818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe730107450 msgr2=0x7fe73019ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe735cf9640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe730107450 0x7fe73019ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe735cf9640 1 -- 172.21.15.19:0/2428396818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73019e3f0 con 0x7fe730106990 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe736cfb640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe730107450 0x7fe73019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe735cf9640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106990 0x7fe730199af0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe72000d770 tx=0x7fe72000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe7277fe640 1 -- 172.21.15.19:0/2428396818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720017070 con 0x7fe730106990 2024-04-25T10:58:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73019e6e0 con 0x7fe730106990 2024-04-25T10:58:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe7277fe640 1 -- 172.21.15.19:0/2428396818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe720009c30 con 0x7fe730106990 2024-04-25T10:58:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe7277fe640 1 -- 172.21.15.19:0/2428396818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72000f620 con 0x7fe730106990 2024-04-25T10:58:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.289+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7301a6060 con 0x7fe730106990 2024-04-25T10:58:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.290+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73006a190 con 0x7fe730106990 2024-04-25T10:58:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.291+0000 7fe7277fe640 1 -- 172.21.15.19:0/2428396818 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe72001a070 con 0x7fe730106990 2024-04-25T10:58:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.292+0000 7fe7277fe640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe71c0afc60 0x7fe71c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.293+0000 7fe7364fa640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe71c0afc60 0x7fe71c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.293+0000 7fe7277fe640 1 -- 172.21.15.19:0/2428396818 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe7200deb20 con 0x7fe730106990 2024-04-25T10:58:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.293+0000 7fe7364fa640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe71c0afc60 0x7fe71c0b2170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005fd0 tx=0x7fe72c009870 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.296+0000 7fe7277fe640 1 -- 172.21.15.19:0/2428396818 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe720009de0 con 0x7fe730106990 2024-04-25T10:58:51.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:51 smithi019 ceph-mon[29209]: pgmap v133: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:51.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.526+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe73010a210 con 0x7fe730106990 2024-04-25T10:58:51.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.527+0000 7fe7277fe640 1 -- 172.21.15.19:0/2428396818 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fe7200a73f0 con 0x7fe730106990 2024-04-25T10:58:51.527 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:58:51.527 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:58:51.528 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe71c0afc60 msgr2=0x7fe71c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe71c0afc60 0x7fe71c0b2170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005fd0 tx=0x7fe72c009870 comp rx=0 tx=0).stop 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106990 msgr2=0x7fe730199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106990 0x7fe730199af0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe72000d770 tx=0x7fe72000dc30 comp rx=0 tx=0).stop 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 shutdown_connections 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe71c0afc60 0x7fe71c0b2170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe730107450 0x7fe73019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe730106990 0x7fe730199af0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 --2- 172.21.15.19:0/2428396818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe730102400 0x7fe730199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 >> 172.21.15.19:0/2428396818 conn(0x7fe730077890 msgr2=0x7fe7300ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 shutdown_connections 2024-04-25T10:58:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.529+0000 7fe73875e640 1 -- 172.21.15.19:0/2428396818 wait complete. 2024-04-25T10:58:51.541 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:58:51.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:51 smithi112 ceph-mon[32799]: pgmap v133: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.655+0000 7ff43326e640 1 -- 172.21.15.19:0/3442890669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c109fc0 msgr2=0x7ff42c103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.655+0000 7ff43326e640 1 --2- 172.21.15.19:0/3442890669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c109fc0 0x7ff42c103c00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009900 tx=0x7ff41c02a410 comp rx=0 tx=0).stop 2024-04-25T10:58:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.655+0000 7ff43326e640 1 -- 172.21.15.19:0/3442890669 shutdown_connections 2024-04-25T10:58:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.655+0000 7ff43326e640 1 --2- 172.21.15.19:0/3442890669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c104270 0x7ff42c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.655+0000 7ff43326e640 1 --2- 172.21.15.19:0/3442890669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c109fc0 0x7ff42c103c00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.655+0000 7ff43326e640 1 --2- 172.21.15.19:0/3442890669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c109b20 0x7ff42c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.655+0000 7ff43326e640 1 -- 172.21.15.19:0/3442890669 >> 172.21.15.19:0/3442890669 conn(0x7ff42c0ff1e0 msgr2=0x7ff42c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.656+0000 7ff43326e640 1 -- 172.21.15.19:0/3442890669 shutdown_connections 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.656+0000 7ff43326e640 1 -- 172.21.15.19:0/3442890669 wait complete. 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.656+0000 7ff43326e640 1 Processor -- start 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.656+0000 7ff43326e640 1 -- start start 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.656+0000 7ff43326e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c104270 0x7ff42c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.656+0000 7ff43326e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c109b20 0x7ff42c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43326e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c109fc0 0x7ff42c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43326e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c113d10 con 0x7ff42c104270 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43326e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c113b90 con 0x7ff42c109b20 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43326e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c113e90 con 0x7ff42c109fc0 2024-04-25T10:58:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c104270 0x7ff42c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c104270 0x7ff42c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49318/0 (socket says 172.21.15.19:49318) 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 -- 172.21.15.19:0/2597590224 learned_addr learned my addr 172.21.15.19:0/2597590224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43180b640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c109fc0 0x7ff42c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 -- 172.21.15.19:0/2597590224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c109fc0 msgr2=0x7ff42c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c109fc0 0x7ff42c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff430809640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c109b20 0x7ff42c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 -- 172.21.15.19:0/2597590224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c109b20 msgr2=0x7ff42c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c109b20 0x7ff42c19acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 -- 172.21.15.19:0/2597590224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c19f5f0 con 0x7ff42c104270 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43100a640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c104270 0x7ff42c19a790 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff42800d740 tx=0x7ff42800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff4227fc640 1 -- 172.21.15.19:0/2597590224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428017070 con 0x7ff42c104270 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff4227fc640 1 -- 172.21.15.19:0/2597590224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42800de50 con 0x7ff42c104270 2024-04-25T10:58:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c19f8e0 con 0x7ff42c104270 2024-04-25T10:58:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff4227fc640 1 -- 172.21.15.19:0/2597590224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428004ce0 con 0x7ff42c104270 2024-04-25T10:58:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.657+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c19fde0 con 0x7ff42c104270 2024-04-25T10:58:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.660+0000 7ff4227fc640 1 -- 172.21.15.19:0/2597590224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff428004e80 con 0x7ff42c104270 2024-04-25T10:58:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.660+0000 7ff43a1fe740 1 Processor -- start 2024-04-25T10:58:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.660+0000 7ff43a1fe740 1 -- start start 2024-04-25T10:58:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa4a4d0 0x55a80fe14c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe25ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff4227fc640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4180afc60 0x7ff4180b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a80fd79d80 con 0x55a80fa4a4d0 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a80fd33a20 con 0x55a80fa526b0 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a80fd75d40 con 0x55a80fa73390 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa4a4d0 0x55a80fe14c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43180b640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4180afc60 0x7ff4180b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa4a4d0 0x55a80fe14c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49326/0 (socket says 172.21.15.19:49326) 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 -- 172.21.15.19:0/847672534 learned_addr learned my addr 172.21.15.19:0/847672534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff430809640 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe25ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff4227fc640 1 -- 172.21.15.19:0/2597590224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff4280dea60 con 0x7ff42c104270 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43180b640 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 -- 172.21.15.19:0/847672534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 msgr2=0x55a80fe146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 -- 172.21.15.19:0/847672534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 msgr2=0x55a80fe25ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe25ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 -- 172.21.15.19:0/847672534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a80fd3ec30 con 0x55a80fa4a4d0 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43180b640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4180afc60 0x7ff4180b2170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff424005fd0 tx=0x7ff424012a30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff43100a640 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa4a4d0 0x55a80fe14c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff4280a68d0 tx=0x7ff4280a66e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f73ba948feef275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.661+0000 7ff4177fe640 1 -- 172.21.15.19:0/847672534 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280a72d0 con 0x55a80fa4a4d0 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff4177fe640 1 -- 172.21.15.19:0/847672534 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4280a6bd0 con 0x55a80fa4a4d0 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff4177fe640 1 -- 172.21.15.19:0/847672534 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280dc600 con 0x55a80fa4a4d0 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 -- 172.21.15.19:0/847672534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa4a4d0 msgr2=0x55a80fe14c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa4a4d0 0x55a80fe14c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff4280a68d0 tx=0x7ff4280a66e0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 -- 172.21.15.19:0/847672534 shutdown_connections 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe25ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa4a4d0 0x55a80fe14c20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/847672534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 -- 172.21.15.19:0/847672534 >> 172.21.15.19:0/847672534 conn(0x55a80fa674e0 msgr2=0x55a80fe10fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.662+0000 7ff43a1fe740 1 -- 172.21.15.19:0/847672534 shutdown_connections 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 -- 172.21.15.19:0/847672534 wait complete. 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 -- start start 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 Processor -- start 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa4a4d0 0x55a80fe23370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe2f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a80fe261d0 con 0x55a80fa526b0 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a80fd00dc0 con 0x55a80fa4a4d0 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a80fe26350 con 0x55a80fa73390 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.663+0000 7ff43a1fe740 10 client.0 _ll_register_callbacks cb 0x55a80fd1c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49340/0 (socket says 172.21.15.19:49340) 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 -- 172.21.15.19:0/743203444 learned_addr learned my addr 172.21.15.19:0/743203444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff43100a640 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa4a4d0 0x55a80fe23370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff43180b640 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe2f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 -- 172.21.15.19:0/743203444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 msgr2=0x55a80fe2f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe2f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 -- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa4a4d0 msgr2=0x55a80fe23370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa4a4d0 0x55a80fe23370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 -- 172.21.15.19:0/743203444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a80fa8b3f0 con 0x55a80fa526b0 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff430809640 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe238b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009900 tx=0x7ff41c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34a0df4988fe174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff414ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff414ff9640 1 -- 172.21.15.19:0/743203444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c040070 con 0x55a80fa526b0 2024-04-25T10:58:51.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff414ff9640 1 -- 172.21.15.19:0/743203444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff41c004040 con 0x55a80fa526b0 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4280a6bd0 con 0x55a80fa526b0 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff414ff9640 1 -- 172.21.15.19:0/743203444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff41c035040 con 0x55a80fa526b0 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff43a1fe740 10 client.16224 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.664+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a80fb557d0 con 0x55a80fa526b0 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff414ff9640 1 -- 172.21.15.19:0/743203444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4280a6bd0 con 0x55a80fa526b0 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff414ff9640 1 -- 172.21.15.19:0/743203444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff41c0047e0 con 0x55a80fa526b0 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff43a1fe740 10 client.16224 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff43a1fe740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff43a1fe740 20 client.16224 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa31f60 0x55a80fe359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff43a1fe740 4 client.16224 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a80fd87990 con 0x55a80fa31f60 2024-04-25T10:58:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.665+0000 7ff43100a640 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa31f60 0x55a80fe359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.666+0000 7ff43100a640 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa31f60 0x55a80fe359a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b637dc7b44c1cb server_cookie=e73787f440a2e7d6 in_seq=0 out_seq=0 2024-04-25T10:58:51.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.666+0000 7ff414ff9640 10 client.16224 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.666+0000 7ff414ff9640 1 -- 172.21.15.19:0/743203444 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a80fd87990 con 0x55a80fa31f60 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.666+0000 7ff414ff9640 10 client.16224 handle_command_reply: tid=0 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.666+0000 7ff43a1fe740 1 client.16224 shutdown 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.666+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 msgr2=0x55a80fe238b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.666+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe238b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009900 tx=0x7ff41c032f20 comp rx=0 tx=0).stop 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 shutdown_connections 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa31f60 0x55a80fe359a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa73390 0x55a80fe2f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe238b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff414ff9640 0 client.16224 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/743203444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa4a4d0 0x55a80fe23370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 >> 172.21.15.19:0/743203444 conn(0x55a80fa674e0 msgr2=0x55a80fe164c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 shutdown_connections 2024-04-25T10:58:51.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 1 -- 172.21.15.19:0/743203444 wait complete. 2024-04-25T10:58:51.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.667+0000 7ff43a1fe740 20 client.16224 trim_cache size 0 max 16384 2024-04-25T10:58:51.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.680+0000 7ff43a1fe740 1 Processor -- start 2024-04-25T10:58:51.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.680+0000 7ff43a1fe740 1 -- start start 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa73390 0x55a80fe3b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1ab800 con 0x55a80fa526b0 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1ae080 con 0x55a80fa73390 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1abe70 con 0x55a80fa4a4d0 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe16b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49348/0 (socket says 172.21.15.19:49348) 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 -- 172.21.15.19:0/3158607167 learned_addr learned my addr 172.21.15.19:0/3158607167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43100a640 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43180b640 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa73390 0x55a80fe3b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 -- 172.21.15.19:0/3158607167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 msgr2=0x55a80fe2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 -- 172.21.15.19:0/3158607167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa73390 msgr2=0x55a80fe3b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa73390 0x55a80fe3b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 -- 172.21.15.19:0/3158607167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a80fe15770 con 0x55a80fa526b0 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff43100a640 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff430809640 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe16b90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff41c004d50 tx=0x7ff41c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff622e4d1016e300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff416ffd640 1 -- 172.21.15.19:0/3158607167 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c040070 con 0x55a80fa526b0 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff416ffd640 1 -- 172.21.15.19:0/3158607167 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c0048c0 con 0x55a80fa526b0 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.681+0000 7ff416ffd640 1 -- 172.21.15.19:0/3158607167 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c04d6b0 con 0x55a80fa526b0 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3158607167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 msgr2=0x55a80fe16b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe16b90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff41c004d50 tx=0x7ff41c02a410 comp rx=0 tx=0).stop 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3158607167 shutdown_connections 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa73390 0x55a80fe3b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa526b0 0x55a80fe16b90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3158607167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3158607167 >> 172.21.15.19:0/3158607167 conn(0x55a80fa674e0 msgr2=0x55a80fe16480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3158607167 shutdown_connections 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3158607167 wait complete. 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.682+0000 7ff43a1fe740 1 -- start start 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 1 Processor -- start 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe27020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa73390 0x55a80fe27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1ac540 con 0x55a80fa73390 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1ad9b0 con 0x55a80fa526b0 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1acc10 con 0x55a80fa4a4d0 2024-04-25T10:58:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43a1fe740 10 client.0 _ll_register_callbacks cb 0x7ff41c032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa73390 0x55a80fe27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa73390 0x55a80fe27670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49352/0 (socket says 172.21.15.19:49352) 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 -- 172.21.15.19:0/3939701614 learned_addr learned my addr 172.21.15.19:0/3939701614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff430809640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe27020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43100a640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 -- 172.21.15.19:0/3939701614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 msgr2=0x55a80fe2bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 -- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 msgr2=0x55a80fe27020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe27020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43180b640 1 -- 172.21.15.19:0/3939701614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a80fe30180 con 0x55a80fa73390 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.683+0000 7ff43100a640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43180b640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa73390 0x55a80fe27670 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff4240109a0 tx=0x7ff424005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e86579331a63c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:51.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff414ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff414ff9640 1 -- 172.21.15.19:0/3939701614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42401cd30 con 0x55a80fa73390 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff414ff9640 1 -- 172.21.15.19:0/3939701614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff42402b480 con 0x55a80fa73390 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff41c0048c0 con 0x55a80fa73390 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff414ff9640 1 -- 172.21.15.19:0/3939701614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff42402b680 con 0x55a80fa73390 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 10 client.16230 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a80fe9dab0 con 0x55a80fa73390 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff414ff9640 1 -- 172.21.15.19:0/3939701614 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c0048c0 con 0x55a80fa73390 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff414ff9640 1 -- 172.21.15.19:0/3939701614 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff424020400 con 0x55a80fa73390 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 10 client.16230 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 20 client.16230 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa073a0 0x55a80fea0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 4 client.16230 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:58:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.684+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a80fd81800 con 0x55a80fa073a0 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.685+0000 7ff43100a640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa073a0 0x55a80fea0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.685+0000 7ff414ff9640 10 client.16230 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.685+0000 7ff43100a640 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa073a0 0x55a80fea0280 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab95a00d2a55403 server_cookie=bf7dd4133de819c0 in_seq=0 out_seq=0 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.685+0000 7ff414ff9640 1 -- 172.21.15.19:0/3939701614 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a80fd81800 con 0x55a80fa073a0 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.685+0000 7ff414ff9640 10 client.16230 handle_command_reply: tid=0 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.685+0000 7ff43a1fe740 1 client.16230 shutdown 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa73390 msgr2=0x55a80fe27670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa73390 0x55a80fe27670 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff4240109a0 tx=0x7ff424005bd0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 shutdown_connections 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a80fa073a0 0x55a80fea0280 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a80fa73390 0x55a80fe27670 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a80fa526b0 0x55a80fe27020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff414ff9640 0 client.16230 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:51.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 --2- 172.21.15.19:0/3939701614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80fa4a4d0 0x55a80fe2bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 >> 172.21.15.19:0/3939701614 conn(0x55a80fa674e0 msgr2=0x55a80fe16480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 shutdown_connections 2024-04-25T10:58:51.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 1 -- 172.21.15.19:0/3939701614 wait complete. 2024-04-25T10:58:51.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43a1fe740 20 client.16230 trim_cache size 0 max 16384 2024-04-25T10:58:51.693 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:58:51.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4180afc60 msgr2=0x7ff4180b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4180afc60 0x7ff4180b2170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff424005fd0 tx=0x7ff424012a30 comp rx=0 tx=0).stop 2024-04-25T10:58:51.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c104270 msgr2=0x7ff42c19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:51.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c104270 0x7ff42c19a790 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff42800d740 tx=0x7ff42800dc00 comp rx=0 tx=0).stop 2024-04-25T10:58:51.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 shutdown_connections 2024-04-25T10:58:51.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4180afc60 0x7ff4180b2170 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff42c109fc0 0x7ff42c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c109b20 0x7ff42c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 --2- 172.21.15.19:0/2597590224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff42c104270 0x7ff42c19a790 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.686+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 >> 172.21.15.19:0/2597590224 conn(0x7ff42c0ff1e0 msgr2=0x7ff42c10f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.687+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 shutdown_connections 2024-04-25T10:58:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:51.687+0000 7ff43326e640 1 -- 172.21.15.19:0/2597590224 wait complete. 2024-04-25T10:58:51.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:51 smithi169 ceph-mon[33049]: pgmap v133: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:52.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:52 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2428396818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:58:52.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:52 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2428396818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:58:52.698 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:58:52.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:58:52.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:58:52.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:52 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2428396818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:58:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 -- 172.21.15.19:0/174309896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 msgr2=0x7fe4241036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 --2- 172.21.15.19:0/174309896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe4241036c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe420009900 tx=0x7fe42002a410 comp rx=0 tx=0).stop 2024-04-25T10:58:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 -- 172.21.15.19:0/174309896 shutdown_connections 2024-04-25T10:58:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 --2- 172.21.15.19:0/174309896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe424104270 0x7fe424110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 --2- 172.21.15.19:0/174309896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe424109fc0 0x7fe424103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 --2- 172.21.15.19:0/174309896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe4241036c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 -- 172.21.15.19:0/174309896 >> 172.21.15.19:0/174309896 conn(0x7fe4240ff1e0 msgr2=0x7fe424101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 -- 172.21.15.19:0/174309896 shutdown_connections 2024-04-25T10:58:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.829+0000 7fe42b651640 1 -- 172.21.15.19:0/174309896 wait complete. 2024-04-25T10:58:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 Processor -- start 2024-04-25T10:58:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 -- start start 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe424104270 0x7fe42419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe42419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe424109fc0 0x7fe42419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe424113d10 con 0x7fe424109b20 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe424113b90 con 0x7fe424109fc0 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe42b651640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe424113e90 con 0x7fe424104270 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe428bec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe42419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe428bec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe42419acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58998/0 (socket says 172.21.15.19:58998) 2024-04-25T10:58:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.830+0000 7fe428bec640 1 -- 172.21.15.19:0/705265491 learned_addr learned my addr 172.21.15.19:0/705265491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe4293ed640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe424104270 0x7fe42419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe429bee640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe424109fc0 0x7fe42419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe428bec640 1 -- 172.21.15.19:0/705265491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe424104270 msgr2=0x7fe42419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe428bec640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe424104270 0x7fe42419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe428bec640 1 -- 172.21.15.19:0/705265491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe424109fc0 msgr2=0x7fe42419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe428bec640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe424109fc0 0x7fe42419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe428bec640 1 -- 172.21.15.19:0/705265491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42419f5d0 con 0x7fe424109b20 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe429bee640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe424109fc0 0x7fe42419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe428bec640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe42419acd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe414005fd0 tx=0x7fe41400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe41a7fc640 1 -- 172.21.15.19:0/705265491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414017070 con 0x7fe424109b20 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe41a7fc640 1 -- 172.21.15.19:0/705265491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe414009c30 con 0x7fe424109b20 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42419f8c0 con 0x7fe424109b20 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe41a7fc640 1 -- 172.21.15.19:0/705265491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414003d10 con 0x7fe424109b20 2024-04-25T10:58:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.831+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241a7240 con 0x7fe424109b20 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 Processor -- start 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe41a7fc640 1 -- 172.21.15.19:0/705265491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe414003ef0 con 0x7fe424109b20 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 -- start start 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dd16410 0x558a3e0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dced550 0x558a3e0b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dcf5730 0x558a3e0b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a3e01cb00 con 0x558a3dced550 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558a3dfb8920 con 0x558a3dcf5730 2024-04-25T10:58:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4325e7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558a3e018ac0 con 0x558a3dd16410 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4293ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dced550 0x558a3e0b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4293ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dced550 0x558a3e0b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59000/0 (socket says 172.21.15.19:59000) 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe4293ed640 1 -- 172.21.15.19:0/2884154283 learned_addr learned my addr 172.21.15.19:0/2884154283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.834+0000 7fe429bee640 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dd16410 0x558a3e0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe428bec640 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dcf5730 0x558a3e0b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe41a7fc640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe4100afce0 0x7fe4100b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe4293ed640 1 -- 172.21.15.19:0/2884154283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dd16410 msgr2=0x558a3e0b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe4293ed640 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dd16410 0x558a3e0b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe4293ed640 1 -- 172.21.15.19:0/2884154283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dcf5730 msgr2=0x558a3e0b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe4293ed640 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dcf5730 0x558a3e0b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe4293ed640 1 -- 172.21.15.19:0/2884154283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a3ddf67e0 con 0x558a3dced550 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe429bee640 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dd16410 0x558a3e0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe428bec640 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dcf5730 0x558a3e0b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe429bee640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe4100afce0 0x7fe4100b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe41a7fc640 1 -- 172.21.15.19:0/705265491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe4140dec40 con 0x7fe424109b20 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe4293ed640 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dced550 0x558a3e0b6a30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe420009900 tx=0x7fe420032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83b0e27f95359ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe40f7fe640 1 -- 172.21.15.19:0/2884154283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42003f070 con 0x558a3dced550 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe40f7fe640 1 -- 172.21.15.19:0/2884154283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe420032e40 con 0x558a3dced550 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.835+0000 7fe40f7fe640 1 -- 172.21.15.19:0/2884154283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420035070 con 0x558a3dced550 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 -- 172.21.15.19:0/2884154283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dced550 msgr2=0x558a3e0b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dced550 0x558a3e0b6a30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe420009900 tx=0x7fe420032c50 comp rx=0 tx=0).stop 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe429bee640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe4100afce0 0x7fe4100b21f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005fd0 tx=0x7fe41c004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 -- 172.21.15.19:0/2884154283 shutdown_connections 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dcf5730 0x558a3e0b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dced550 0x558a3e0b6a30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2884154283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dd16410 0x558a3e0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 -- 172.21.15.19:0/2884154283 >> 172.21.15.19:0/2884154283 conn(0x558a3dd0a560 msgr2=0x558a3e0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 -- 172.21.15.19:0/2884154283 shutdown_connections 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.836+0000 7fe4325e7740 1 -- 172.21.15.19:0/2884154283 wait complete. 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 -- start start 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 Processor -- start 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dced550 0x558a3e0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dcf5730 0x558a3e0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dd16410 0x558a3e0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a3e0c7da0 con 0x558a3dcf5730 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558a3dfb65d0 con 0x558a3dd16410 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558a3e0c7f20 con 0x558a3dced550 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4325e7740 10 client.0 _ll_register_callbacks cb 0x558a3dfa4080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe428bec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dcf5730 0x558a3e0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe428bec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dcf5730 0x558a3e0c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59012/0 (socket says 172.21.15.19:59012) 2024-04-25T10:58:52.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe428bec640 1 -- 172.21.15.19:0/2002142416 learned_addr learned my addr 172.21.15.19:0/2002142416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe429bee640 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dd16410 0x558a3e0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.837+0000 7fe4293ed640 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dced550 0x558a3e0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe428bec640 1 -- 172.21.15.19:0/2002142416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dced550 msgr2=0x558a3e0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe428bec640 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dced550 0x558a3e0c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe428bec640 1 -- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dd16410 msgr2=0x558a3e0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe428bec640 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dd16410 0x558a3e0d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe428bec640 1 -- 172.21.15.19:0/2002142416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a3de56f70 con 0x558a3dcf5730 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe428bec640 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dcf5730 0x558a3e0c55a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe4140a6b10 tx=0x7fe4140a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d2061415d3b9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe40cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe40cff9640 1 -- 172.21.15.19:0/2002142416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4140a7510 con 0x558a3dcf5730 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe40cff9640 1 -- 172.21.15.19:0/2002142416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe4140a30a0 con 0x558a3dcf5730 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558a3e02a710 con 0x558a3dcf5730 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe40cff9640 1 -- 172.21.15.19:0/2002142416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe414009de0 con 0x558a3dcf5730 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe4325e7740 10 client.16242 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:52.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a3dcee170 con 0x558a3dcf5730 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.838+0000 7fe40cff9640 1 -- 172.21.15.19:0/2002142416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558a3e02a710 con 0x558a3dcf5730 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe40cff9640 1 -- 172.21.15.19:0/2002142416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe4140be030 con 0x558a3dcf5730 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4325e7740 10 client.16242 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4325e7740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4325e7740 20 client.16242 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcd51f0 0x558a3e0d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4325e7740 4 client.16242 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a3e024580 con 0x558a3dcd51f0 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4293ed640 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcd51f0 0x558a3e0d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe40cff9640 10 client.16242 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:52.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.839+0000 7fe4293ed640 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcd51f0 0x558a3e0d74f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459353208020f6dc server_cookie=99a639747975a253 in_seq=0 out_seq=0 2024-04-25T10:58:52.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe40cff9640 1 -- 172.21.15.19:0/2002142416 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a3e024580 con 0x558a3dcd51f0 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe40cff9640 10 client.16242 handle_command_reply: tid=0 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 client.16242 shutdown 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dcf5730 msgr2=0x558a3e0c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dcf5730 0x558a3e0c55a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe4140a6b10 tx=0x7fe4140a6760 comp rx=0 tx=0).stop 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 shutdown_connections 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcd51f0 0x558a3e0d74f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dd16410 0x558a3e0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe40cff9640 0 client.16242 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dcf5730 0x558a3e0c55a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 --2- 172.21.15.19:0/2002142416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dced550 0x558a3e0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.840+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 >> 172.21.15.19:0/2002142416 conn(0x558a3dd0a560 msgr2=0x558a3e0b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.841+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 shutdown_connections 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.841+0000 7fe4325e7740 1 -- 172.21.15.19:0/2002142416 wait complete. 2024-04-25T10:58:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.841+0000 7fe4325e7740 20 client.16242 trim_cache size 0 max 16384 2024-04-25T10:58:52.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 Processor -- start 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 -- start start 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241ab600 con 0x558a3dd16410 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241ade60 con 0x558a3dced550 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4325e7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241abc50 con 0x558a3dcf5730 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe429bee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe429bee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0dd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59022/0 (socket says 172.21.15.19:59022) 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe429bee640 1 -- 172.21.15.19:0/1644905965 learned_addr learned my addr 172.21.15.19:0/1644905965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe428bec640 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.859+0000 7fe4293ed640 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe429bee640 1 -- 172.21.15.19:0/1644905965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 msgr2=0x558a3e0dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe429bee640 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe429bee640 1 -- 172.21.15.19:0/1644905965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 msgr2=0x558a3e0b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe429bee640 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe429bee640 1 -- 172.21.15.19:0/1644905965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a3e0b39a0 con 0x558a3dd16410 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe429bee640 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0dd1f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe41c013840 tx=0x7fe41c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e41afef7c409af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe40effd640 1 -- 172.21.15.19:0/1644905965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c01c9c0 con 0x558a3dd16410 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe40effd640 1 -- 172.21.15.19:0/1644905965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41c01cb60 con 0x558a3dd16410 2024-04-25T10:58:52.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.860+0000 7fe40effd640 1 -- 172.21.15.19:0/1644905965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c01a670 con 0x558a3dd16410 2024-04-25T10:58:52.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 -- 172.21.15.19:0/1644905965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 msgr2=0x558a3e0dd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0dd1f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe41c013840 tx=0x7fe41c016000 comp rx=0 tx=0).stop 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 -- 172.21.15.19:0/1644905965 shutdown_connections 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0dd1f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1644905965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 -- 172.21.15.19:0/1644905965 >> 172.21.15.19:0/1644905965 conn(0x558a3dd0a560 msgr2=0x558a3e0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 -- 172.21.15.19:0/1644905965 shutdown_connections 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 -- 172.21.15.19:0/1644905965 wait complete. 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 -- start start 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.861+0000 7fe4325e7740 1 Processor -- start 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241ac320 con 0x558a3dd16410 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241ad790 con 0x558a3dced550 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241ac9f0 con 0x558a3dcf5730 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 10 client.0 _ll_register_callbacks cb 0x7fe4140a2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59028/0 (socket says 172.21.15.19:59028) 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 -- 172.21.15.19:0/1243918943 learned_addr learned my addr 172.21.15.19:0/1243918943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe428bec640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4293ed640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 -- 172.21.15.19:0/1243918943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 msgr2=0x558a3e0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 -- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 msgr2=0x558a3e0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 -- 172.21.15.19:0/1243918943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a3e13e860 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe428bec640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe429bee640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0c9850 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005c00 tx=0x7fe41c027730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e033bc394c3648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe40cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe40cff9640 1 -- 172.21.15.19:0/1243918943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c030640 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe40cff9640 1 -- 172.21.15.19:0/1243918943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe41c01a410 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.862+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41c01cb60 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe40cff9640 1 -- 172.21.15.19:0/1243918943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe41c0307e0 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 10 client.16248 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a3e13fa80 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe40cff9640 1 -- 172.21.15.19:0/1243918943 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41c01cb60 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe40cff9640 1 -- 172.21.15.19:0/1243918943 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe41c02e070 con 0x558a3dd16410 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 10 client.16248 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:52.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 20 client.16248 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcaa630 0x558a3e142100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 4 client.16248 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a3df9fac0 con 0x558a3dcaa630 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.863+0000 7fe4293ed640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcaa630 0x558a3e142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.864+0000 7fe40cff9640 10 client.16248 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.864+0000 7fe4293ed640 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcaa630 0x558a3e142100 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ef47f25cfe4afd server_cookie=f5647ced5afd7300 in_seq=0 out_seq=0 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe40cff9640 1 -- 172.21.15.19:0/1243918943 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a3df9fac0 con 0x558a3dcaa630 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe40cff9640 10 client.16248 handle_command_reply: tid=0 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 client.16248 shutdown 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 msgr2=0x558a3e0c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0c9850 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005c00 tx=0x7fe41c027730 comp rx=0 tx=0).stop 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 shutdown_connections 2024-04-25T10:58:52.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558a3dcaa630 0x558a3e142100 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe40cff9640 0 client.16248 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a3dd16410 0x558a3e0c9850 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558a3dcf5730 0x558a3e0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 --2- 172.21.15.19:0/1243918943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a3dced550 0x558a3e0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 >> 172.21.15.19:0/1243918943 conn(0x558a3dd0a560 msgr2=0x558a3e0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 shutdown_connections 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 1 -- 172.21.15.19:0/1243918943 wait complete. 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.865+0000 7fe4325e7740 20 client.16248 trim_cache size 0 max 16384 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:58:52.871 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "4e146d76-1602-4987-bb96-c163ef6e458b", 2024-04-25T10:58:52.872 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:58:52.872 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:58:52.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe4100afce0 msgr2=0x7fe4100b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe4100afce0 0x7fe4100b21f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005fd0 tx=0x7fe41c004a40 comp rx=0 tx=0).stop 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 msgr2=0x7fe42419acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe42419acd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe414005fd0 tx=0x7fe41400dde0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 shutdown_connections 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe4100afce0 0x7fe4100b21f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe424109fc0 0x7fe42419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe424109b20 0x7fe42419acd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 --2- 172.21.15.19:0/705265491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe424104270 0x7fe42419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 >> 172.21.15.19:0/705265491 conn(0x7fe4240ff1e0 msgr2=0x7fe4240ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 shutdown_connections 2024-04-25T10:58:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.866+0000 7fe42b651640 1 -- 172.21.15.19:0/705265491 wait complete. 2024-04-25T10:58:52.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:58:52.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:58:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 -- 172.21.15.19:0/1031513087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73981027c0 msgr2=0x7f7398110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 --2- 172.21.15.19:0/1031513087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73981027c0 0x7f7398110610 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7390009ca0 tx=0x7f739002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 -- 172.21.15.19:0/1031513087 shutdown_connections 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 --2- 172.21.15.19:0/1031513087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73981027c0 0x7f7398110610 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 --2- 172.21.15.19:0/1031513087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7398109fc0 0x7f7398102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 --2- 172.21.15.19:0/1031513087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7398109b20 0x7f7398101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 -- 172.21.15.19:0/1031513087 >> 172.21.15.19:0/1031513087 conn(0x7f7398077880 msgr2=0x7f7398077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 -- 172.21.15.19:0/1031513087 shutdown_connections 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 -- 172.21.15.19:0/1031513087 wait complete. 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 Processor -- start 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.992+0000 7f739d4bf640 1 -- start start 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f739d4bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73981027c0 0x7f739819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f739d4bf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7398109b20 0x7f739819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f739d4bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7398109fc0 0x7f739819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f739d4bf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7398113ce0 con 0x7f7398109fc0 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f739d4bf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7398113b60 con 0x7f73981027c0 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f739d4bf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7398113e60 con 0x7f7398109b20 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7398109fc0 0x7f739819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7398109fc0 0x7f739819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59040/0 (socket says 172.21.15.19:59040) 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 -- 172.21.15.19:0/2446778020 learned_addr learned my addr 172.21.15.19:0/2446778020 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73981027c0 0x7f739819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f738ffff640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7398109b20 0x7f739819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 -- 172.21.15.19:0/2446778020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7398109b20 msgr2=0x7f739819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7398109b20 0x7f739819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 -- 172.21.15.19:0/2446778020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73981027c0 msgr2=0x7f739819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73981027c0 0x7f739819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 -- 172.21.15.19:0/2446778020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739819f5a0 con 0x7f7398109fc0 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f738ffff640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7398109b20 0x7f739819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73981027c0 0x7f739819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f73977fe640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7398109fc0 0x7f739819ef60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7390009430 tx=0x7f7390031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f7394ff9640 1 -- 172.21.15.19:0/2446778020 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390003b70 con 0x7f7398109fc0 2024-04-25T10:58:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f7394ff9640 1 -- 172.21.15.19:0/2446778020 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7390003d10 con 0x7f7398109fc0 2024-04-25T10:58:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739819f830 con 0x7f7398109fc0 2024-04-25T10:58:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.993+0000 7f7394ff9640 1 -- 172.21.15.19:0/2446778020 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739002fa50 con 0x7f7398109fc0 2024-04-25T10:58:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.994+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981a7210 con 0x7f7398109fc0 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.996+0000 7f73a4370740 1 Processor -- start 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.996+0000 7f7394ff9640 1 -- 172.21.15.19:0/2446778020 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f739002fbf0 con 0x7f7398109fc0 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.996+0000 7f73a4370740 1 -- start start 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f73a4370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x563353441a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f73a4370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530784d0 0x563353441fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f73a4370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335344e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f73a4370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5633533a7d80 con 0x5633530784d0 2024-04-25T10:58:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f73a4370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633533618b0 con 0x5633530a1390 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f73a4370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5633533a3d40 con 0x5633530806b0 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530784d0 0x563353441fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530784d0 0x563353441fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59050/0 (socket says 172.21.15.19:59050) 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 -- 172.21.15.19:0/2212541079 learned_addr learned my addr 172.21.15.19:0/2212541079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f738ffff640 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x563353441a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f738ffff640 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335344e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 -- 172.21.15.19:0/2212541079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 msgr2=0x56335344e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335344e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 -- 172.21.15.19:0/2212541079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 msgr2=0x563353441a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x563353441a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7394ff9640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f73840afce0 0x7f73840b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 -- 172.21.15.19:0/2212541079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563353332a40 con 0x5633530784d0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f73977fe640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f73840afce0 0x7f73840b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530784d0 0x563353441fa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7388009d40 tx=0x7f73880117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1297efcc9fa14b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.997+0000 7f7394ff9640 1 -- 172.21.15.19:0/2446778020 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f73901002a0 con 0x7f7398109fc0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.998+0000 7f738d7fa640 1 -- 172.21.15.19:0/2212541079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388012650 con 0x5633530784d0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.998+0000 7f738d7fa640 1 -- 172.21.15.19:0/2212541079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7388009e30 con 0x5633530784d0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.998+0000 7f738d7fa640 1 -- 172.21.15.19:0/2212541079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738800fc30 con 0x5633530784d0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.998+0000 7f73a4370740 1 -- 172.21.15.19:0/2212541079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530784d0 msgr2=0x563353441fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.998+0000 7f73a4370740 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530784d0 0x563353441fa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7388009d40 tx=0x7f73880117b0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.998+0000 7f73977fe640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f73840afce0 0x7f73840b21f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f73900fdbe0 tx=0x7f7390097080 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- 172.21.15.19:0/2212541079 shutdown_connections 2024-04-25T10:58:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335344e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530784d0 0x563353441fa0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 --2- 172.21.15.19:0/2212541079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x563353441a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- 172.21.15.19:0/2212541079 >> 172.21.15.19:0/2212541079 conn(0x5633530954e0 msgr2=0x56335343e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- 172.21.15.19:0/2212541079 shutdown_connections 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- 172.21.15.19:0/2212541079 wait complete. 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- start start 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 Processor -- start 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530784d0 0x56335344c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x56335344cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x5633534524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56335344efe0 con 0x5633530806b0 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56335332ed10 con 0x5633530a1390 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:52.999+0000 7f73a4370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56335344f160 con 0x5633530784d0 2024-04-25T10:58:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f73a4370740 10 client.0 _ll_register_callbacks cb 0x56335335b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x56335344cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x56335344cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59058/0 (socket says 172.21.15.19:59058) 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 -- 172.21.15.19:0/2912698268 learned_addr learned my addr 172.21.15.19:0/2912698268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f738ffff640 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530784d0 0x56335344c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f738ffff640 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x5633534524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 -- 172.21.15.19:0/2912698268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530784d0 msgr2=0x56335344c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530784d0 0x56335344c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 -- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 msgr2=0x5633534524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x5633534524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 -- 172.21.15.19:0/2912698268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633531837d0 con 0x5633530806b0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x56335344cba0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f738800f770 tx=0x7f73880058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732768e17411740c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f737e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f737e7fc640 1 -- 172.21.15.19:0/2912698268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388028020 con 0x5633530806b0 2024-04-25T10:58:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f737e7fc640 1 -- 172.21.15.19:0/2912698268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f7388005240 con 0x5633530806b0 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.000+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7388009e30 con 0x5633530806b0 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f737e7fc640 1 -- 172.21.15.19:0/2912698268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7388033050 con 0x5633530806b0 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 10 client.16260 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5633530b93f0 con 0x5633530806b0 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f737e7fc640 1 -- 172.21.15.19:0/2912698268 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7388009e30 con 0x5633530806b0 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f737e7fc640 1 -- 172.21.15.19:0/2912698268 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f7388012450 con 0x5633530806b0 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 10 client.16260 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:53.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 20 client.16260 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56335305ff60 0x56335345e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 4 client.16260 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633533b5990 con 0x56335305ff60 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.001+0000 7f738ffff640 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56335305ff60 0x56335345e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.002+0000 7f737e7fc640 10 client.16260 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.002+0000 7f738ffff640 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56335305ff60 0x56335345e670 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1962293a946db5 server_cookie=3141acbad86054e1 in_seq=0 out_seq=0 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.002+0000 7f737e7fc640 1 -- 172.21.15.19:0/2912698268 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633533b5990 con 0x56335305ff60 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.002+0000 7f737e7fc640 10 client.16260 handle_command_reply: tid=0 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 client.16260 shutdown 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 msgr2=0x56335344cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x56335344cba0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f738800f770 tx=0x7f73880058f0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 shutdown_connections 2024-04-25T10:58:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56335305ff60 0x56335345e670 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530a1390 0x5633534524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x56335344cba0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f737e7fc640 0 client.16260 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 --2- 172.21.15.19:0/2912698268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530784d0 0x56335344c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 >> 172.21.15.19:0/2912698268 conn(0x5633530954e0 msgr2=0x563353443780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 shutdown_connections 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 1 -- 172.21.15.19:0/2912698268 wait complete. 2024-04-25T10:58:53.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.003+0000 7f73a4370740 20 client.16260 trim_cache size 0 max 16384 2024-04-25T10:58:53.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.021+0000 7f73a4370740 1 Processor -- start 2024-04-25T10:58:53.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.021+0000 7f73a4370740 1 -- start start 2024-04-25T10:58:53.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.021+0000 7f73a4370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x56335344b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f73a4370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335343d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f73a4370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530a1390 0x56335343df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f73a4370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ad8e0 con 0x5633530a1390 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f73a4370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73981b0160 con 0x5633530784d0 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f73a4370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73981adf50 con 0x5633530806b0 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530a1390 0x56335343df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530a1390 0x56335343df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59062/0 (socket says 172.21.15.19:59062) 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 -- 172.21.15.19:0/2720142185 learned_addr learned my addr 172.21.15.19:0/2720142185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f7396ffd640 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335343d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x56335344b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 -- 172.21.15.19:0/2720142185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 msgr2=0x56335343d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335343d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 -- 172.21.15.19:0/2720142185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 msgr2=0x56335344b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x56335344b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 -- 172.21.15.19:0/2720142185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633530790f0 con 0x5633530a1390 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738ffff640 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530a1390 0x56335343df10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f73980ffc20 tx=0x7f7380009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1182ee5d114a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738cff9640 1 -- 172.21.15.19:0/2720142185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800126f0 con 0x5633530a1390 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738cff9640 1 -- 172.21.15.19:0/2720142185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f738002d070 con 0x5633530a1390 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.022+0000 7f738cff9640 1 -- 172.21.15.19:0/2720142185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380003af0 con 0x5633530a1390 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 -- 172.21.15.19:0/2720142185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530a1390 msgr2=0x56335343df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530a1390 0x56335343df10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f73980ffc20 tx=0x7f7380009600 comp rx=0 tx=0).stop 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 -- 172.21.15.19:0/2720142185 shutdown_connections 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530a1390 0x56335343df10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530806b0 0x56335343d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 --2- 172.21.15.19:0/2720142185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x56335344b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 -- 172.21.15.19:0/2720142185 >> 172.21.15.19:0/2720142185 conn(0x5633530954e0 msgr2=0x563353443cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 -- 172.21.15.19:0/2720142185 shutdown_connections 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.023+0000 7f73a4370740 1 -- 172.21.15.19:0/2720142185 wait complete. 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 -- start start 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 Processor -- start 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x563353458d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x563353456110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530a1390 0x563353456650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ae620 con 0x5633530806b0 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73981afa90 con 0x5633530784d0 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73981aecf0 con 0x5633530a1390 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f73a4370740 10 client.0 _ll_register_callbacks cb 0x7f73800099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x563353456110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x563353456110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59074/0 (socket says 172.21.15.19:59074) 2024-04-25T10:58:53.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 -- 172.21.15.19:0/1244670253 learned_addr learned my addr 172.21.15.19:0/1244670253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f738ffff640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x563353458d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f738ffff640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530a1390 0x563353456650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 -- 172.21.15.19:0/1244670253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530a1390 msgr2=0x563353456650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530a1390 0x563353456650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 -- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 msgr2=0x563353458d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x563353458d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f7396ffd640 1 -- 172.21.15.19:0/1244670253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633533684b0 con 0x5633530806b0 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.024+0000 7f738ffff640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x563353458d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f7396ffd640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x563353456110 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7388009ad0 tx=0x7f738800f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96379176db825707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f737e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f737e7fc640 1 -- 172.21.15.19:0/1244670253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388028040 con 0x5633530806b0 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f737e7fc640 1 -- 172.21.15.19:0/1244670253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f7388012c90 con 0x5633530806b0 2024-04-25T10:58:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f738002d070 con 0x5633530806b0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f737e7fc640 1 -- 172.21.15.19:0/1244670253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7388045050 con 0x5633530806b0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 10 client.16266 fetch_fsmap learned FSMap version 20 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5633534c63d0 con 0x5633530806b0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f737e7fc640 1 -- 172.21.15.19:0/1244670253 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f738002d070 con 0x5633530806b0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f737e7fc640 1 -- 172.21.15.19:0/1244670253 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f7388004660 con 0x5633530806b0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 10 client.16266 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 20 client.16266 populate_metadata read hostname 'smithi019' 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5633530353a0 0x5633534c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 4 client.16266 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.025+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5633533af800 con 0x5633530353a0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f738ffff640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5633530353a0 0x5633534c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f737e7fc640 10 client.16266 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f738ffff640 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5633530353a0 0x5633534c8ac0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80fc2c2453a9db7 server_cookie=c055ab4bef6e7c19 in_seq=0 out_seq=0 2024-04-25T10:58:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f737e7fc640 1 -- 172.21.15.19:0/1244670253 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5633533af800 con 0x5633530353a0 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f737e7fc640 10 client.16266 handle_command_reply: tid=0 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 client.16266 shutdown 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 msgr2=0x563353456110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x563353456110 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7388009ad0 tx=0x7f738800f6e0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 shutdown_connections 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5633530353a0 0x5633534c8ac0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5633530a1390 0x563353456650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633530806b0 0x563353456110 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 --2- 172.21.15.19:0/1244670253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633530784d0 0x563353458d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f737e7fc640 0 client.16266 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.026+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 >> 172.21.15.19:0/1244670253 conn(0x5633530954e0 msgr2=0x563353443cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 shutdown_connections 2024-04-25T10:58:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f73a4370740 1 -- 172.21.15.19:0/1244670253 wait complete. 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f73a4370740 20 client.16266 trim_cache size 0 max 16384 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: "4e146d76-1602-4987-bb96-c163ef6e458b": { 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "4e146d76-1602-4987-bb96-c163ef6e458b", 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:58:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:58:53.040 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:58:53.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f73840afce0 msgr2=0x7f73840b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:53.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f73840afce0 0x7f73840b21f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f73900fdbe0 tx=0x7f7390097080 comp rx=0 tx=0).stop 2024-04-25T10:58:53.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7398109fc0 msgr2=0x7f739819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:53.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7398109fc0 0x7f739819ef60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7390009430 tx=0x7f7390031660 comp rx=0 tx=0).stop 2024-04-25T10:58:53.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 shutdown_connections 2024-04-25T10:58:53.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f73840afce0 0x7f73840b21f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7398109fc0 0x7f739819ef60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7398109b20 0x7f739819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 --2- 172.21.15.19:0/2446778020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73981027c0 0x7f739819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:53.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 >> 172.21.15.19:0/2446778020 conn(0x7f7398077880 msgr2=0x7f7398100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:53.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 shutdown_connections 2024-04-25T10:58:53.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:58:53.027+0000 7f739d4bf640 1 -- 172.21.15.19:0/2446778020 wait complete. 2024-04-25T10:58:53.042 INFO:tasks.cephfs.filesystem:scrub status for tag:4e146d76-1602-4987-bb96-c163ef6e458b - {'path': '/', 'tag': '4e146d76-1602-4987-bb96-c163ef6e458b', 'options': 'recursive,force'} 2024-04-25T10:58:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:53 smithi019 ceph-mon[29209]: pgmap v134: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:53 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:58:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:53 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:53 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:58:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:53 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:53.562 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:53 smithi112 ceph-mon[32799]: pgmap v134: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:53.562 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:53 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:58:53.562 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:53 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:53.562 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:53 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:58:53.562 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:53 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:53 smithi169 ceph-mon[33049]: pgmap v134: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:58:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:53 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:58:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:53 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:53 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:58:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:53 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:58:55.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:55 smithi112 ceph-mon[32799]: pgmap v135: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:55.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:55 smithi019 ceph-mon[29209]: pgmap v135: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:55.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:55 smithi169 ceph-mon[33049]: pgmap v135: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:58:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:57 smithi112 ceph-mon[32799]: pgmap v136: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:57.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:57 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:58:57.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:57 smithi019 ceph-mon[29209]: pgmap v136: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:57 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:58:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:57 smithi169 ceph-mon[33049]: pgmap v136: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:58:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:57 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:58:59.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:58:59 smithi112 ceph-mon[32799]: pgmap v137: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:59.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:58:59 smithi019 ceph-mon[29209]: pgmap v137: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:58:59.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:58:59 smithi169 ceph-mon[33049]: pgmap v137: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:01.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:01 smithi112 ceph-mon[32799]: pgmap v138: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:01.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:01 smithi019 ceph-mon[29209]: pgmap v138: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:01.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:01 smithi169 ceph-mon[33049]: pgmap v138: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:03.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:03 smithi112 ceph-mon[32799]: pgmap v139: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T10:59:03.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:03 smithi019 ceph-mon[29209]: pgmap v139: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T10:59:03.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:03 smithi169 ceph-mon[33049]: pgmap v139: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T10:59:05.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:05 smithi112 ceph-mon[32799]: pgmap v140: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:59:05.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:05 smithi019 ceph-mon[29209]: pgmap v140: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:59:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:05 smithi169 ceph-mon[33049]: pgmap v140: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:59:07.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:07 smithi112 ceph-mon[32799]: pgmap v141: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T10:59:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:07 smithi019 ceph-mon[29209]: pgmap v141: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T10:59:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:07 smithi169 ceph-mon[33049]: pgmap v141: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T10:59:09.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:09 smithi112 ceph-mon[32799]: pgmap v142: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:59:09.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:09 smithi019 ceph-mon[29209]: pgmap v142: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:59:09.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:09 smithi169 ceph-mon[33049]: pgmap v142: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:59:11.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:11 smithi019 ceph-mon[29209]: pgmap v143: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:59:11.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:11 smithi169 ceph-mon[33049]: pgmap v143: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:59:11.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:11 smithi112 ceph-mon[32799]: pgmap v143: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T10:59:13.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:13 smithi019 ceph-mon[29209]: pgmap v144: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:59:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:13 smithi169 ceph-mon[33049]: pgmap v144: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:59:13.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:13 smithi112 ceph-mon[32799]: pgmap v144: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T10:59:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:15 smithi019 ceph-mon[29209]: pgmap v145: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:59:15.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:15 smithi169 ceph-mon[33049]: pgmap v145: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:59:15.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:15 smithi112 ceph-mon[32799]: pgmap v145: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:59:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:17 smithi019 ceph-mon[29209]: pgmap v146: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:59:17.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:17 smithi169 ceph-mon[33049]: pgmap v146: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:59:17.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:17 smithi112 ceph-mon[32799]: pgmap v146: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T10:59:19.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:19 smithi019 ceph-mon[29209]: pgmap v147: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:59:19.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:19 smithi169 ceph-mon[33049]: pgmap v147: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:59:19.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:19 smithi112 ceph-mon[32799]: pgmap v147: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:59:21.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:21 smithi019 ceph-mon[29209]: pgmap v148: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:59:21.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:21 smithi169 ceph-mon[33049]: pgmap v148: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:59:21.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:21 smithi112 ceph-mon[32799]: pgmap v148: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T10:59:23.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:59:23.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:59:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 -- 172.21.15.19:0/3791457827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108a10 msgr2=0x7feef8102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 --2- 172.21.15.19:0/3791457827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108a10 0x7feef8102610 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7feee8009900 tx=0x7feee802a410 comp rx=0 tx=0).stop 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 -- 172.21.15.19:0/3791457827 shutdown_connections 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 --2- 172.21.15.19:0/3791457827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef81031c0 0x7feef810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 --2- 172.21.15.19:0/3791457827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feef8108eb0 0x7feef8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 --2- 172.21.15.19:0/3791457827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108a10 0x7feef8102610 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 -- 172.21.15.19:0/3791457827 >> 172.21.15.19:0/3791457827 conn(0x7feef8078110 msgr2=0x7feef8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 -- 172.21.15.19:0/3791457827 shutdown_connections 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.233+0000 7feefe2d1640 1 -- 172.21.15.19:0/3791457827 wait complete. 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.234+0000 7feefe2d1640 1 Processor -- start 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.234+0000 7feefe2d1640 1 -- start start 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.234+0000 7feefe2d1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef81031c0 0x7feef819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.234+0000 7feefe2d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feef8108a10 0x7feef819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.234+0000 7feefe2d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108eb0 0x7feef81a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefe2d1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feef8112c60 con 0x7feef8108eb0 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefe2d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feef8112ae0 con 0x7feef8108a10 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefe2d1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feef8112de0 con 0x7feef81031c0 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108eb0 0x7feef81a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108eb0 0x7feef81a65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53106/0 (socket says 172.21.15.19:53106) 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 -- 172.21.15.19:0/2965118808 learned_addr learned my addr 172.21.15.19:0/2965118808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feef77fe640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feef8108a10 0x7feef819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feef7fff640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef81031c0 0x7feef819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 -- 172.21.15.19:0/2965118808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef81031c0 msgr2=0x7feef819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef81031c0 0x7feef819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 -- 172.21.15.19:0/2965118808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feef8108a10 msgr2=0x7feef819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feef8108a10 0x7feef819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 -- 172.21.15.19:0/2965118808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef819c2e0 con 0x7feef8108eb0 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feef77fe640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feef8108a10 0x7feef819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefc86e640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108eb0 0x7feef81a65a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7feef000eaa0 tx=0x7feef000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feef57fa640 1 -- 172.21.15.19:0/2965118808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef000cdb0 con 0x7feef8108eb0 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feef57fa640 1 -- 172.21.15.19:0/2965118808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef0005020 con 0x7feef8108eb0 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feef57fa640 1 -- 172.21.15.19:0/2965118808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef00116d0 con 0x7feef8108eb0 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.235+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef81a6ba0 con 0x7feef8108eb0 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.236+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef81a70f0 con 0x7feef8108eb0 2024-04-25T10:59:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 Processor -- start 2024-04-25T10:59:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7feef57fa640 1 -- 172.21.15.19:0/2965118808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7feef00051d0 con 0x7feef8108eb0 2024-04-25T10:59:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 -- start start 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e1aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a596b0 0x555dc9e278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555dc9d80d80 con 0x555dc9a7a390 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555dc9d3a8b0 con 0x555dc9a596b0 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.238+0000 7fef0517e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555dc9d7cd40 con 0x555dc9a514d0 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e1aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e1aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53112/0 (socket says 172.21.15.19:53112) 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 -- 172.21.15.19:0/2521646521 learned_addr learned my addr 172.21.15.19:0/2521646521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef7fff640 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef57fa640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7feee00afce0 0x7feee00b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a596b0 0x555dc9e278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 -- 172.21.15.19:0/2521646521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 msgr2=0x555dc9e1afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 -- 172.21.15.19:0/2521646521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a596b0 msgr2=0x555dc9e278f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a596b0 0x555dc9e278f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 -- 172.21.15.19:0/2521646521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc9d0ba40 con 0x555dc9a7a390 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feefc86e640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7feee00afce0 0x7feee00b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef57fa640 1 -- 172.21.15.19:0/2965118808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7feef00df240 con 0x7feef8108eb0 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feef77fe640 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e1aa60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feeec00bfc0 tx=0x7feeec00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe6dca59ba6856d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feede7fc640 1 -- 172.21.15.19:0/2521646521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec019030 con 0x555dc9a7a390 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feede7fc640 1 -- 172.21.15.19:0/2521646521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeec0035f0 con 0x555dc9a7a390 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.239+0000 7feede7fc640 1 -- 172.21.15.19:0/2521646521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec011720 con 0x555dc9a7a390 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 -- 172.21.15.19:0/2521646521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 msgr2=0x555dc9e1aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7feefc86e640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7feee00afce0 0x7feee00b21f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feef00a3490 tx=0x7feef0060040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e1aa60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feeec00bfc0 tx=0x7feeec00b650 comp rx=0 tx=0).stop 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 -- 172.21.15.19:0/2521646521 shutdown_connections 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a596b0 0x555dc9e278f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 --2- 172.21.15.19:0/2521646521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e1aa60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 -- 172.21.15.19:0/2521646521 >> 172.21.15.19:0/2521646521 conn(0x555dc9a6e4e0 msgr2=0x555dc9e17210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 -- 172.21.15.19:0/2521646521 shutdown_connections 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.240+0000 7fef0517e740 1 -- 172.21.15.19:0/2521646521 wait complete. 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 -- start start 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 Processor -- start 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a514d0 0x555dc9e255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e25b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555dc9e27fe0 con 0x555dc9a7a390 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555dc9d07d10 con 0x555dc9a514d0 2024-04-25T10:59:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555dc9e28160 con 0x555dc9a596b0 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7fef0517e740 10 client.0 _ll_register_callbacks cb 0x555dc9d34210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7feef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7feef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e2b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53124/0 (socket says 172.21.15.19:53124) 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.241+0000 7feef77fe640 1 -- 172.21.15.19:0/3019654928 learned_addr learned my addr 172.21.15.19:0/3019654928 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef7fff640 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e25b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef77fe640 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a514d0 0x555dc9e255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef77fe640 1 -- 172.21.15.19:0/3019654928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 msgr2=0x555dc9e25b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef77fe640 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e25b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef77fe640 1 -- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a514d0 msgr2=0x555dc9e255d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef77fe640 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a514d0 0x555dc9e255d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef77fe640 1 -- 172.21.15.19:0/3019654928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc9b5c7d0 con 0x555dc9a7a390 2024-04-25T10:59:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feef77fe640 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e2b430 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x555dc9e24c20 tx=0x7feeec005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195f54f29e0742ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feed77fe640 1 -- 172.21.15.19:0/3019654928 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec022070 con 0x555dc9a7a390 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feed77fe640 1 -- 172.21.15.19:0/3019654928 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7feeec003fe0 con 0x555dc9a7a390 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec0035f0 con 0x555dc9a7a390 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7feed77fe640 1 -- 172.21.15.19:0/3019654928 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feeec034050 con 0x555dc9a7a390 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7fef0517e740 10 client.16278 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.242+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555dc9a923f0 con 0x555dc9a7a390 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7feed77fe640 1 -- 172.21.15.19:0/3019654928 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeec0035f0 con 0x555dc9a7a390 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7feed77fe640 1 -- 172.21.15.19:0/3019654928 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7feeec01e460 con 0x555dc9a7a390 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7fef0517e740 10 client.16278 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7fef0517e740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7fef0517e740 20 client.16278 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7fef0517e740 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a38f60 0x555dc9e374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7fef0517e740 4 client.16278 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dc9d8e990 con 0x555dc9a38f60 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.243+0000 7feef7fff640 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a38f60 0x555dc9e374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.244+0000 7feed77fe640 10 client.16278 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.244+0000 7feef7fff640 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a38f60 0x555dc9e374c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=280cf08732d34867 server_cookie=4f980beb500e4cd1 in_seq=0 out_seq=0 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.244+0000 7feed77fe640 1 -- 172.21.15.19:0/3019654928 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555dc9d8e990 con 0x555dc9a38f60 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.244+0000 7feed77fe640 10 client.16278 handle_command_reply: tid=0 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.244+0000 7fef0517e740 1 client.16278 shutdown 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 msgr2=0x555dc9e2b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e2b430 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x555dc9e24c20 tx=0x7feeec005850 comp rx=0 tx=0).stop 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 shutdown_connections 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a38f60 0x555dc9e374c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a7a390 0x555dc9e2b430 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e25b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 --2- 172.21.15.19:0/3019654928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a514d0 0x555dc9e255d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7feed77fe640 0 client.16278 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 >> 172.21.15.19:0/3019654928 conn(0x555dc9a6e4e0 msgr2=0x555dc9e1c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 shutdown_connections 2024-04-25T10:59:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 1 -- 172.21.15.19:0/3019654928 wait complete. 2024-04-25T10:59:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.245+0000 7fef0517e740 20 client.16278 trim_cache size 0 max 16384 2024-04-25T10:59:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 Processor -- start 2024-04-25T10:59:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 -- start start 2024-04-25T10:59:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a596b0 0x555dc9e2b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 0x555dc9e2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feef81ad940 con 0x555dc9a596b0 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feef81b01c0 con 0x555dc9a7a390 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7fef0517e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feef81adfb0 con 0x555dc9a514d0 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a596b0 0x555dc9e2b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a596b0 0x555dc9e2b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53138/0 (socket says 172.21.15.19:53138) 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 -- 172.21.15.19:0/2696440446 learned_addr learned my addr 172.21.15.19:0/2696440446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef77fe640 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 0x555dc9e2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef77fe640 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 -- 172.21.15.19:0/2696440446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 msgr2=0x555dc9e1cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 -- 172.21.15.19:0/2696440446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 msgr2=0x555dc9e2b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 0x555dc9e2b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 -- 172.21.15.19:0/2696440446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc9a520f0 con 0x555dc9a596b0 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef77fe640 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.264+0000 7feef7fff640 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a596b0 0x555dc9e2b420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feef80791a0 tx=0x7feee8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a623cacd57223ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.265+0000 7feeddffb640 1 -- 172.21.15.19:0/2696440446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8035040 con 0x555dc9a596b0 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.265+0000 7feeddffb640 1 -- 172.21.15.19:0/2696440446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee8049070 con 0x555dc9a596b0 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.265+0000 7feeddffb640 1 -- 172.21.15.19:0/2696440446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8004920 con 0x555dc9a596b0 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.265+0000 7fef0517e740 1 -- 172.21.15.19:0/2696440446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a596b0 msgr2=0x555dc9e2b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.265+0000 7fef0517e740 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a596b0 0x555dc9e2b420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feef80791a0 tx=0x7feee8009680 comp rx=0 tx=0).stop 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 -- 172.21.15.19:0/2696440446 shutdown_connections 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 0x555dc9e2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a596b0 0x555dc9e2b420 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 --2- 172.21.15.19:0/2696440446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a514d0 0x555dc9e1cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 -- 172.21.15.19:0/2696440446 >> 172.21.15.19:0/2696440446 conn(0x555dc9a6e4e0 msgr2=0x555dc9e1c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 -- 172.21.15.19:0/2696440446 shutdown_connections 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 -- 172.21.15.19:0/2696440446 wait complete. 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 -- start start 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 Processor -- start 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.266+0000 7fef0517e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a514d0 0x555dc9e2ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 0x555dc9e975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feef81ae680 con 0x555dc9a514d0 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feef81afaf0 con 0x555dc9a7a390 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feef81aed50 con 0x555dc9a596b0 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 10 client.0 _ll_register_callbacks cb 0x7feeec013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a514d0 0x555dc9e2ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a514d0 0x555dc9e2ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53152/0 (socket says 172.21.15.19:53152) 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 -- 172.21.15.19:0/3678066397 learned_addr learned my addr 172.21.15.19:0/3678066397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef7fff640 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 -- 172.21.15.19:0/3678066397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 msgr2=0x555dc9e2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 -- 172.21.15.19:0/3678066397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 msgr2=0x555dc9e975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 0x555dc9e975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 -- 172.21.15.19:0/3678066397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc9d414b0 con 0x555dc9a514d0 2024-04-25T10:59:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef7fff640 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feef77fe640 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a514d0 0x555dc9e2ece0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feeec0058c0 tx=0x7feeec0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29dccaec15e3087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feed77fe640 1 -- 172.21.15.19:0/3678066397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec03d060 con 0x555dc9a514d0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feed77fe640 1 -- 172.21.15.19:0/3678066397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7feeec00bc40 con 0x555dc9a514d0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee8049070 con 0x555dc9a514d0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7feed77fe640 1 -- 172.21.15.19:0/3678066397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feeec022070 con 0x555dc9a514d0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 10 client.16284 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.267+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555dc9e9ebd0 con 0x555dc9a514d0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7feed77fe640 1 -- 172.21.15.19:0/3678066397 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee8049070 con 0x555dc9a514d0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7feed77fe640 1 -- 172.21.15.19:0/3678066397 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7feeec003e00 con 0x555dc9a514d0 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7fef0517e740 10 client.16284 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7fef0517e740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7fef0517e740 20 client.16284 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7fef0517e740 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a0e3a0 0x555dc9ea1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7fef0517e740 4 client.16284 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555dc9d88800 con 0x555dc9a0e3a0 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.268+0000 7feef7fff640 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a0e3a0 0x555dc9ea1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7feed77fe640 10 client.16284 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7feef7fff640 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a0e3a0 0x555dc9ea1390 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c98f2a6a86208ac server_cookie=649a05bf657815a5 in_seq=0 out_seq=0 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7feed77fe640 1 -- 172.21.15.19:0/3678066397 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555dc9d88800 con 0x555dc9a0e3a0 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7feed77fe640 10 client.16284 handle_command_reply: tid=0 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 client.16284 shutdown 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a514d0 msgr2=0x555dc9e2ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a514d0 0x555dc9e2ece0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feeec0058c0 tx=0x7feeec0039b0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 shutdown_connections 2024-04-25T10:59:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555dc9a0e3a0 0x555dc9ea1390 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dc9a7a390 0x555dc9e975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dc9a596b0 0x555dc9e2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7feed77fe640 0 client.16284 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 --2- 172.21.15.19:0/3678066397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dc9a514d0 0x555dc9e2ece0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 >> 172.21.15.19:0/3678066397 conn(0x555dc9a6e4e0 msgr2=0x555dc9e23c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 shutdown_connections 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 1 -- 172.21.15.19:0/3678066397 wait complete. 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.269+0000 7fef0517e740 20 client.16284 trim_cache size 0 max 16384 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:59:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:59:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:59:23.281 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7feee00afce0 msgr2=0x7feee00b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7feee00afce0 0x7feee00b21f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feef00a3490 tx=0x7feef0060040 comp rx=0 tx=0).stop 2024-04-25T10:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108eb0 msgr2=0x7feef81a65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108eb0 0x7feef81a65a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7feef000eaa0 tx=0x7feef000ef60 comp rx=0 tx=0).stop 2024-04-25T10:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 shutdown_connections 2024-04-25T10:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7feee00afce0 0x7feee00b21f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feef8108eb0 0x7feef81a65a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feef8108a10 0x7feef819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 --2- 172.21.15.19:0/2965118808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef81031c0 0x7feef819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 >> 172.21.15.19:0/2965118808 conn(0x7feef8078110 msgr2=0x7feef81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 shutdown_connections 2024-04-25T10:59:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.270+0000 7feefe2d1640 1 -- 172.21.15.19:0/2965118808 wait complete. 2024-04-25T10:59:23.283 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:59:23.283 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 -- 172.21.15.19:0/1792714302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 msgr2=0x7fde90102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 --2- 172.21.15.19:0/1792714302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde90102ee0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009900 tx=0x7fde8c02a410 comp rx=0 tx=0).stop 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 -- 172.21.15.19:0/1792714302 shutdown_connections 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 --2- 172.21.15.19:0/1792714302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90107b20 0x7fde9010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 --2- 172.21.15.19:0/1792714302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde90107060 0x7fde90107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 --2- 172.21.15.19:0/1792714302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde90102ee0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 -- 172.21.15.19:0/1792714302 >> 172.21.15.19:0/1792714302 conn(0x7fde900fdf10 msgr2=0x7fde90100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 -- 172.21.15.19:0/1792714302 shutdown_connections 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.396+0000 7fde9659d640 1 -- 172.21.15.19:0/1792714302 wait complete. 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 Processor -- start 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 -- start start 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde9019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90107060 0x7fde9019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde90107b20 0x7fde901a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde90112a30 con 0x7fde90102ad0 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde901128b0 con 0x7fde90107b20 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9659d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fde90112bb0 con 0x7fde90107060 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9559b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde9019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9559b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde9019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53160/0 (socket says 172.21.15.19:53160) 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde9559b640 1 -- 172.21.15.19:0/172423094 learned_addr learned my addr 172.21.15.19:0/172423094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde95d9c640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde90107b20 0x7fde901a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.397+0000 7fde94d9a640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90107060 0x7fde9019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9559b640 1 -- 172.21.15.19:0/172423094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90107060 msgr2=0x7fde9019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9559b640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90107060 0x7fde9019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9559b640 1 -- 172.21.15.19:0/172423094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde90107b20 msgr2=0x7fde901a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9559b640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde90107b20 0x7fde901a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9559b640 1 -- 172.21.15.19:0/172423094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9019c110 con 0x7fde90102ad0 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9559b640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde9019b350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005940 tx=0x7fde8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde867fc640 1 -- 172.21.15.19:0/172423094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c03f030 con 0x7fde90102ad0 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde901a6970 con 0x7fde90102ad0 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde867fc640 1 -- 172.21.15.19:0/172423094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c003cb0 con 0x7fde90102ad0 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde867fc640 1 -- 172.21.15.19:0/172423094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c035070 con 0x7fde90102ad0 2024-04-25T10:59:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.398+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde901a6e60 con 0x7fde90102ad0 2024-04-25T10:59:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.399+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9006a030 con 0x7fde90102ad0 2024-04-25T10:59:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.400+0000 7fde867fc640 1 -- 172.21.15.19:0/172423094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fde8c042030 con 0x7fde90102ad0 2024-04-25T10:59:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.401+0000 7fde867fc640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fde7c0afc50 0x7fde7c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.402+0000 7fde94d9a640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fde7c0afc50 0x7fde7c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.402+0000 7fde867fc640 1 -- 172.21.15.19:0/172423094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fde8c0ffaa0 con 0x7fde90102ad0 2024-04-25T10:59:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.402+0000 7fde94d9a640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fde7c0afc50 0x7fde7c0b2160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fde901065e0 tx=0x7fde8000b410 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.406+0000 7fde867fc640 1 -- 172.21.15.19:0/172423094 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde8c0c7d20 con 0x7fde90102ad0 2024-04-25T10:59:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:23 smithi019 ceph-mon[29209]: pgmap v149: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:59:23.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.637+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde9006bad0 con 0x7fde90102ad0 2024-04-25T10:59:23.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.638+0000 7fde867fc640 1 -- 172.21.15.19:0/172423094 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fde9006bad0 con 0x7fde90102ad0 2024-04-25T10:59:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:59:23.638 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:59:23.639 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fde7c0afc50 msgr2=0x7fde7c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fde7c0afc50 0x7fde7c0b2160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fde901065e0 tx=0x7fde8000b410 comp rx=0 tx=0).stop 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 msgr2=0x7fde9019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde9019b350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005940 tx=0x7fde8c0042f0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 shutdown_connections 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fde7c0afc50 0x7fde7c0b2160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde90107b20 0x7fde901a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90107060 0x7fde9019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 --2- 172.21.15.19:0/172423094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90102ad0 0x7fde9019b350 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 >> 172.21.15.19:0/172423094 conn(0x7fde900fdf10 msgr2=0x7fde9010e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 shutdown_connections 2024-04-25T10:59:23.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.640+0000 7fde9659d640 1 -- 172.21.15.19:0/172423094 wait complete. 2024-04-25T10:59:23.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:59:23.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:23 smithi169 ceph-mon[33049]: pgmap v149: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/4275585699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db81069b0 msgr2=0x7f2db8106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/4275585699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db81069b0 0x7f2db8106de0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009900 tx=0x7f2da002a410 comp rx=0 tx=0).stop 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/4275585699 shutdown_connections 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/4275585699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2db8107470 0x7f2db810ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/4275585699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db81069b0 0x7f2db8106de0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/4275585699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db8102420 0x7f2db8102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/4275585699 >> 172.21.15.19:0/4275585699 conn(0x7f2db8077920 msgr2=0x7f2db8077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/4275585699 shutdown_connections 2024-04-25T10:59:23.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.766+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/4275585699 wait complete. 2024-04-25T10:59:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.767+0000 7f2dbcfc0640 1 Processor -- start 2024-04-25T10:59:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.767+0000 7f2dbcfc0640 1 -- start start 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2dbcfc0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db8102420 0x7f2db8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2dbcfc0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db81069b0 0x7f2db8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2dbcfc0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2db8107470 0x7f2db819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2dbcfc0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8111b50 con 0x7f2db8102420 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2dbcfc0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81119d0 con 0x7f2db8107470 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2dbcfc0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8111cd0 con 0x7f2db81069b0 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db8102420 0x7f2db8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db8102420 0x7f2db8198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53178/0 (socket says 172.21.15.19:53178) 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 -- 172.21.15.19:0/807741839 learned_addr learned my addr 172.21.15.19:0/807741839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db6d9d640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2db8107470 0x7f2db819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2daffff640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db81069b0 0x7f2db8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 -- 172.21.15.19:0/807741839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db81069b0 msgr2=0x7f2db8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db81069b0 0x7f2db8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 -- 172.21.15.19:0/807741839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2db8107470 msgr2=0x7f2db819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2db8107470 0x7f2db819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 -- 172.21.15.19:0/807741839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db819d3d0 con 0x7f2db8102420 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2daffff640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db81069b0 0x7f2db8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2db659c640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db8102420 0x7f2db8198570 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e9f0 tx=0x7f2da800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2daf7fe640 1 -- 172.21.15.19:0/807741839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da800cd80 con 0x7f2db8102420 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db819d6c0 con 0x7f2db8102420 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2daf7fe640 1 -- 172.21.15.19:0/807741839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da80054d0 con 0x7f2db8102420 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.768+0000 7f2daf7fe640 1 -- 172.21.15.19:0/807741839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8010690 con 0x7f2db8102420 2024-04-25T10:59:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.769+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db819dbc0 con 0x7f2db8102420 2024-04-25T10:59:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.771+0000 7f2daf7fe640 1 -- 172.21.15.19:0/807741839 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f2da8004e90 con 0x7f2db8102420 2024-04-25T10:59:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.771+0000 7f2dc3e71740 1 Processor -- start 2024-04-25T10:59:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.771+0000 7f2dc3e71740 1 -- start start 2024-04-25T10:59:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee63c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeaa16b0 0x560abee74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560abedc8d80 con 0x560abea994d0 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560abed82a20 con 0x560abeac2390 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560abedc4d40 con 0x560abeaa16b0 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee63c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee63c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53190/0 (socket says 172.21.15.19:53190) 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 -- 172.21.15.19:0/2730397274 learned_addr learned my addr 172.21.15.19:0/2730397274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2daf7fe640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f2da40afc60 0x7f2da40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2daffff640 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeaa16b0 0x560abee74ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db6d9d640 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db6d9d640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f2da40afc60 0x7f2da40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2daf7fe640 1 -- 172.21.15.19:0/807741839 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2da80df570 con 0x7f2db8102420 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 -- 172.21.15.19:0/2730397274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeaa16b0 msgr2=0x560abee74ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeaa16b0 0x560abee74ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 -- 172.21.15.19:0/2730397274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 msgr2=0x560abee636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2db659c640 1 -- 172.21.15.19:0/2730397274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abed8dc30 con 0x560abea994d0 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.772+0000 7f2daffff640 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeaa16b0 0x560abee74ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.773+0000 7f2db659c640 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee63c00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2da80a7440 tx=0x7f2da8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384611ab1f4c9b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.773+0000 7f2db6d9d640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f2da40afc60 0x7f2da40b2170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005c40 tx=0x7f2db000d040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.773+0000 7f2dacff9640 1 -- 172.21.15.19:0/2730397274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80a7510 con 0x560abea994d0 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.773+0000 7f2dacff9640 1 -- 172.21.15.19:0/2730397274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da80a7250 con 0x560abea994d0 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.773+0000 7f2dacff9640 1 -- 172.21.15.19:0/2730397274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8005180 con 0x560abea994d0 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2730397274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 msgr2=0x560abee63c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee63c00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2da80a7440 tx=0x7f2da8017000 comp rx=0 tx=0).stop 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2730397274 shutdown_connections 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeaa16b0 0x560abee74ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee63c00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2730397274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2730397274 >> 172.21.15.19:0/2730397274 conn(0x560abeab64e0 msgr2=0x560abee5ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2730397274 shutdown_connections 2024-04-25T10:59:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2730397274 wait complete. 2024-04-25T10:59:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- start start 2024-04-25T10:59:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 Processor -- start 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abea994d0 0x560abee72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abeaa16b0 0x560abee72b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee7e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560abee75140 con 0x560abeaa16b0 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560abed4fdc0 con 0x560abeac2390 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.774+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560abee752c0 con 0x560abea994d0 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2dc3e71740 10 client.0 _ll_register_callbacks cb 0x560abed6b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abeaa16b0 0x560abee72b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abeaa16b0 0x560abee72b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53198/0 (socket says 172.21.15.19:53198) 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 -- 172.21.15.19:0/1376887454 learned_addr learned my addr 172.21.15.19:0/1376887454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2db6d9d640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2db659c640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abea994d0 0x560abee72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 -- 172.21.15.19:0/1376887454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abea994d0 msgr2=0x560abee72630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abea994d0 0x560abee72630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 -- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 msgr2=0x560abee7e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee7e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 -- 172.21.15.19:0/1376887454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abeada3f0 con 0x560abeaa16b0 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2db659c640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abea994d0 0x560abee72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2db6d9d640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee7e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2daffff640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abeaa16b0 0x560abee72b70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2da0031440 tx=0x7f2da00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527ed0d8bd88c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2d9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2d9dffb640 1 -- 172.21.15.19:0/1376887454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0035040 con 0x560abeaa16b0 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2d9dffb640 1 -- 172.21.15.19:0/1376887454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2da0005450 con 0x560abeaa16b0 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.775+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2da80a7250 con 0x560abeaa16b0 2024-04-25T10:59:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2d9dffb640 1 -- 172.21.15.19:0/1376887454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2da0005650 con 0x560abeaa16b0 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 10 client.16302 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560abeba47d0 con 0x560abeaa16b0 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2d9dffb640 1 -- 172.21.15.19:0/1376887454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da80a7250 con 0x560abeaa16b0 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2d9dffb640 1 -- 172.21.15.19:0/1376887454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2da0003950 con 0x560abeaa16b0 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 10 client.16302 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 20 client.16302 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea80f60 0x560abee84a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 4 client.16302 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abedd6990 con 0x560abea80f60 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.776+0000 7f2db659c640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea80f60 0x560abee84a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.777+0000 7f2db659c640 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea80f60 0x560abee84a00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c419ae9ed3caeef4 server_cookie=7bc3436131ee25e1 in_seq=0 out_seq=0 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.777+0000 7f2d9dffb640 10 client.16302 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.777+0000 7f2d9dffb640 1 -- 172.21.15.19:0/1376887454 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560abedd6990 con 0x560abea80f60 2024-04-25T10:59:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.777+0000 7f2d9dffb640 10 client.16302 handle_command_reply: tid=0 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.777+0000 7f2dc3e71740 1 client.16302 shutdown 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abeaa16b0 msgr2=0x560abee72b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abeaa16b0 0x560abee72b70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2da0031440 tx=0x7f2da00042e0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 shutdown_connections 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea80f60 0x560abee84a00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeac2390 0x560abee7e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abeaa16b0 0x560abee72b70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/1376887454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abea994d0 0x560abee72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2d9dffb640 0 client.16302 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 >> 172.21.15.19:0/1376887454 conn(0x560abeab64e0 msgr2=0x560abee654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 shutdown_connections 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 1 -- 172.21.15.19:0/1376887454 wait complete. 2024-04-25T10:59:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.778+0000 7f2dc3e71740 20 client.16302 trim_cache size 0 max 16384 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.791+0000 7f2dc3e71740 1 Processor -- start 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.791+0000 7f2dc3e71740 1 -- start start 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee70cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee71230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 0x560abee8a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81ab820 con 0x560abea994d0 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81ae0a0 con 0x560abeaa16b0 2024-04-25T10:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81abe90 con 0x560abeac2390 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee70cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee70cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53212/0 (socket says 172.21.15.19:53212) 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 -- 172.21.15.19:0/2135265429 learned_addr learned my addr 172.21.15.19:0/2135265429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 -- 172.21.15.19:0/2135265429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 msgr2=0x560abee8a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 0x560abee8a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 -- 172.21.15.19:0/2135265429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 msgr2=0x560abee71230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2daffff640 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee71230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee71230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 -- 172.21.15.19:0/2135265429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abee70970 con 0x560abea994d0 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2db659c640 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee70cf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2da80dd2f0 tx=0x7f2da8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582e1a5914d31a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2d9ffff640 1 -- 172.21.15.19:0/2135265429 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80a6eb0 con 0x560abea994d0 2024-04-25T10:59:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2d9ffff640 1 -- 172.21.15.19:0/2135265429 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da80a3d70 con 0x560abea994d0 2024-04-25T10:59:23.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.792+0000 7f2d9ffff640 1 -- 172.21.15.19:0/2135265429 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80a7090 con 0x560abea994d0 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2135265429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 msgr2=0x560abee70cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee70cf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2da80dd2f0 tx=0x7f2da8004a90 comp rx=0 tx=0).stop 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2135265429 shutdown_connections 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 0x560abee8a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee71230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2135265429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee70cf0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2135265429 >> 172.21.15.19:0/2135265429 conn(0x560abeab64e0 msgr2=0x560abee635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2135265429 shutdown_connections 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.793+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2135265429 wait complete. 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 -- start start 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 Processor -- start 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee7a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee77d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 0x560abee78280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81ac560 con 0x560abea994d0 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81ad9d0 con 0x560abeaa16b0 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81acc30 con 0x560abeac2390 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2dc3e71740 10 client.0 _ll_register_callbacks cb 0x7f2da8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee7a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee7a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53224/0 (socket says 172.21.15.19:53224) 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 -- 172.21.15.19:0/2473339919 learned_addr learned my addr 172.21.15.19:0/2473339919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2daffff640 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee77d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db6d9d640 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 0x560abee78280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 -- 172.21.15.19:0/2473339919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 msgr2=0x560abee78280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 0x560abee78280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 -- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 msgr2=0x560abee77d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee77d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 -- 172.21.15.19:0/2473339919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abee78840 con 0x560abea994d0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2db659c640 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee7a9d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009250 tx=0x7f2da80a72f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5809292c22f6574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.794+0000 7f2d9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2d9dffb640 1 -- 172.21.15.19:0/2473339919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80a7090 con 0x560abea994d0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2d9dffb640 1 -- 172.21.15.19:0/2473339919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2da80a39d0 con 0x560abea994d0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560abedd0800 con 0x560abea994d0 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2d9dffb640 1 -- 172.21.15.19:0/2473339919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2da80a7510 con 0x560abea994d0 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 10 client.16308 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560abeeecad0 con 0x560abea994d0 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2d9dffb640 1 -- 172.21.15.19:0/2473339919 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560abedd0800 con 0x560abea994d0 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2d9dffb640 1 -- 172.21.15.19:0/2473339919 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2da80bcb90 con 0x560abea994d0 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 10 client.16308 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 20 client.16308 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea563a0 0x560abeeeecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 4 client.16308 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560abed4b610 con 0x560abea563a0 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.795+0000 7f2daffff640 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea563a0 0x560abeeeecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2daffff640 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea563a0 0x560abeeeecb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed12bde89e9ca405 server_cookie=a3cb5a9568252345 in_seq=0 out_seq=0 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2d9dffb640 10 client.16308 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2d9dffb640 1 -- 172.21.15.19:0/2473339919 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560abed4b610 con 0x560abea563a0 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2d9dffb640 10 client.16308 handle_command_reply: tid=0 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 client.16308 shutdown 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 msgr2=0x560abee7a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee7a9d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009250 tx=0x7f2da80a72f0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 shutdown_connections 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x560abea563a0 0x560abeeeecb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560abeac2390 0x560abee78280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560abeaa16b0 0x560abee77d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2d9dffb640 0 client.16308 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 --2- 172.21.15.19:0/2473339919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abea994d0 0x560abee7a9d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 >> 172.21.15.19:0/2473339919 conn(0x560abeab64e0 msgr2=0x560abee635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 shutdown_connections 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 1 -- 172.21.15.19:0/2473339919 wait complete. 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.796+0000 7f2dc3e71740 20 client.16308 trim_cache size 0 max 16384 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f2da40afc60 msgr2=0x7f2da40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f2da40afc60 0x7f2da40b2170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005c40 tx=0x7f2db000d040 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db8102420 msgr2=0x7f2db8198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db8102420 0x7f2db8198570 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e9f0 tx=0x7f2da800edf0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 shutdown_connections 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f2da40afc60 0x7f2da40b2170 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2db8107470 0x7f2db819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db81069b0 0x7f2db8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 --2- 172.21.15.19:0/807741839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2db8102420 0x7f2db8198570 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 >> 172.21.15.19:0/807741839 conn(0x7f2db8077920 msgr2=0x7f2db8100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 shutdown_connections 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:23.797+0000 7f2dbcfc0640 1 -- 172.21.15.19:0/807741839 wait complete. 2024-04-25T10:59:23.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:23 smithi112 ceph-mon[32799]: pgmap v149: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T10:59:24.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/172423094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:59:24.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:24 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/172423094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:59:24.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:24 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/172423094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:59:24.809 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:59:24.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:24.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:59:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.939+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3658088205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4108ed0 msgr2=0x7fd5e4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.939+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3658088205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4108ed0 0x7fd5e4102b20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009900 tx=0x7fd5d402a410 comp rx=0 tx=0).stop 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3658088205 shutdown_connections 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3658088205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e4103190 0x7fd5e410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3658088205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4108ed0 0x7fd5e4102b20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3658088205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5e4108a30 0x7fd5e41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3658088205 >> 172.21.15.19:0/3658088205 conn(0x7fd5e40fe260 msgr2=0x7fd5e4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3658088205 shutdown_connections 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3658088205 wait complete. 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 Processor -- start 2024-04-25T10:59:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.940+0000 7fd5eaf4d640 1 -- start start 2024-04-25T10:59:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5eaf4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4103190 0x7fd5e419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5eaf4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5e4108a30 0x7fd5e419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5eaf4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e4108ed0 0x7fd5e419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5eaf4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112ca0 con 0x7fd5e4103190 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5eaf4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112b20 con 0x7fd5e4108a30 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5eaf4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112e20 con 0x7fd5e4108ed0 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4103190 0x7fd5e419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4103190 0x7fd5e419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53246/0 (socket says 172.21.15.19:53246) 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/3103580335 learned_addr learned my addr 172.21.15.19:0/3103580335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5e4108a30 0x7fd5e419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e4108ed0 0x7fd5e419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/3103580335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e4108ed0 msgr2=0x7fd5e419c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e4108ed0 0x7fd5e419c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/3103580335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5e4108a30 msgr2=0x7fd5e419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5e4108a30 0x7fd5e419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/3103580335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e41a6970 con 0x7fd5e4103190 2024-04-25T10:59:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.941+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4103190 0x7fd5e419b660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000d950 tx=0x7fd5e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.942+0000 7fd5d9ffb640 1 -- 172.21.15.19:0/3103580335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0017070 con 0x7fd5e4103190 2024-04-25T10:59:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.942+0000 7fd5d9ffb640 1 -- 172.21.15.19:0/3103580335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e0009c30 con 0x7fd5e4103190 2024-04-25T10:59:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.942+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e41a6c60 con 0x7fd5e4103190 2024-04-25T10:59:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.942+0000 7fd5d9ffb640 1 -- 172.21.15.19:0/3103580335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00047a0 con 0x7fd5e4103190 2024-04-25T10:59:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.942+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a7180 con 0x7fd5e4103190 2024-04-25T10:59:24.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.944+0000 7fd5d9ffb640 1 -- 172.21.15.19:0/3103580335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd5e0004020 con 0x7fd5e4103190 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.944+0000 7fd5f1e02740 1 Processor -- start 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5f1e02740 1 -- start start 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce89807410 0x55ce89ba7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897de550 0x55ce89ba7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce89b0db00 con 0x55ce89807410 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce89aa9920 con 0x55ce897e6730 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ce89b09ac0 con 0x55ce897de550 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce89807410 0x55ce89ba7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce89807410 0x55ce89ba7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53258/0 (socket says 172.21.15.19:53258) 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5d9ffb640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd5cc0afc60 0x7fd5cc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 -- 172.21.15.19:0/3115803444 learned_addr learned my addr 172.21.15.19:0/3115803444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd5cc0afc60 0x7fd5cc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897de550 0x55ce89ba7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5d9ffb640 1 -- 172.21.15.19:0/3103580335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd5e00debc0 con 0x7fd5e4103190 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 -- 172.21.15.19:0/3115803444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897de550 msgr2=0x55ce89ba7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897de550 0x55ce89ba7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 -- 172.21.15.19:0/3115803444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 msgr2=0x55ce89ba80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e94ea640 1 -- 172.21.15.19:0/3115803444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce898e77e0 con 0x55ce89807410 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.945+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897de550 0x55ce89ba7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.946+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce89807410 0x55ce89ba7620 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc014b30 tx=0x7fd5dc012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1507990af7b7c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.946+0000 7fd5caffd640 1 -- 172.21.15.19:0/3115803444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc01ed90 con 0x55ce89807410 2024-04-25T10:59:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.946+0000 7fd5caffd640 1 -- 172.21.15.19:0/3115803444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5dc023070 con 0x55ce89807410 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.946+0000 7fd5caffd640 1 -- 172.21.15.19:0/3115803444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00fc30 con 0x55ce89807410 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3115803444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce89807410 msgr2=0x55ce89ba7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd5cc0afc60 0x7fd5cc0b2170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc022000 tx=0x7fd5dc020000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce89807410 0x55ce89ba7620 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc014b30 tx=0x7fd5dc012390 comp rx=0 tx=0).stop 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3115803444 shutdown_connections 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897de550 0x55ce89ba7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3115803444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce89807410 0x55ce89ba7620 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3115803444 >> 172.21.15.19:0/3115803444 conn(0x55ce897fb560 msgr2=0x55ce89ba3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3115803444 shutdown_connections 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3115803444 wait complete. 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 -- start start 2024-04-25T10:59:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.947+0000 7fd5f1e02740 1 Processor -- start 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bb1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89bb2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce89807410 0x55ce89bbde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce89bb4970 con 0x55ce897de550 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce89aa75d0 con 0x55ce897e6730 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ce89bb4af0 con 0x55ce89807410 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5f1e02740 10 client.0 _ll_register_callbacks cb 0x55ce89a95080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bb1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bb1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53272/0 (socket says 172.21.15.19:53272) 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/1431044969 learned_addr learned my addr 172.21.15.19:0/1431044969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89bb2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce89807410 0x55ce89bbde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/1431044969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce89807410 msgr2=0x55ce89bbde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce89807410 0x55ce89bbde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 msgr2=0x55ce89bb2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89bb2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/1431044969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce89947f70 con 0x55ce897de550 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89bb2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bb1bd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00dc940 tx=0x7fd5e0005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0af053242436c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.948+0000 7fd5c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5c3fff640 1 -- 172.21.15.19:0/1431044969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00a6500 con 0x55ce897de550 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5c3fff640 1 -- 172.21.15.19:0/1431044969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd5e00a3020 con 0x55ce897de550 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5dc023070 con 0x55ce897de550 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5c3fff640 1 -- 172.21.15.19:0/1431044969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd5e00a68a0 con 0x55ce897de550 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 10 client.16320 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ce897df170 con 0x55ce897de550 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5c3fff640 1 -- 172.21.15.19:0/1431044969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5dc023070 con 0x55ce897de550 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5c3fff640 1 -- 172.21.15.19:0/1431044969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd5e0020ad0 con 0x55ce897de550 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 10 client.16320 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 20 client.16320 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce897c61f0 0x55ce89bc3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 4 client.16320 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.949+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce89b1b710 con 0x55ce897c61f0 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.950+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce897c61f0 0x55ce89bc3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.950+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce897c61f0 0x55ce89bc3f30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fb8278b878afbd server_cookie=d7612588784d205f in_seq=0 out_seq=0 2024-04-25T10:59:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.950+0000 7fd5c3fff640 10 client.16320 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5c3fff640 1 -- 172.21.15.19:0/1431044969 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce89b1b710 con 0x55ce897c61f0 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5c3fff640 10 client.16320 handle_command_reply: tid=0 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 client.16320 shutdown 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 msgr2=0x55ce89bb1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bb1bd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00dc940 tx=0x7fd5e0005100 comp rx=0 tx=0).stop 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 shutdown_connections 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce897c61f0 0x55ce89bc3f30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce89807410 0x55ce89bbde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89bb2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/1431044969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bb1bd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5c3fff640 0 client.16320 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 >> 172.21.15.19:0/1431044969 conn(0x55ce897fb560 msgr2=0x55ce89ba7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 shutdown_connections 2024-04-25T10:59:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 1 -- 172.21.15.19:0/1431044969 wait complete. 2024-04-25T10:59:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.951+0000 7fd5f1e02740 20 client.16320 trim_cache size 0 max 16384 2024-04-25T10:59:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 Processor -- start 2024-04-25T10:59:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 -- start start 2024-04-25T10:59:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89ba2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41ad980 con 0x55ce897de550 2024-04-25T10:59:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41b0200 con 0x55ce897e6730 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41adff0 con 0x55ce897c61f0 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53276/0 (socket says 172.21.15.19:53276) 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 -- 172.21.15.19:0/3313022903 learned_addr learned my addr 172.21.15.19:0/3313022903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89ba2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 -- 172.21.15.19:0/3313022903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 msgr2=0x55ce89ba2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89ba2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 -- 172.21.15.19:0/3313022903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 msgr2=0x55ce89ba7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 -- 172.21.15.19:0/3313022903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce89baff10 con 0x55ce897de550 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.970+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bac0e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4005ab0 tx=0x7fd5d4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e6433514f4e8fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5ca7fc640 1 -- 172.21.15.19:0/3313022903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4035040 con 0x55ce897de550 2024-04-25T10:59:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5ca7fc640 1 -- 172.21.15.19:0/3313022903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d404a070 con 0x55ce897de550 2024-04-25T10:59:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5ca7fc640 1 -- 172.21.15.19:0/3313022903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40455e0 con 0x55ce897de550 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3313022903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 msgr2=0x55ce89bac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bac0e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4005ab0 tx=0x7fd5d4005a20 comp rx=0 tx=0).stop 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3313022903 shutdown_connections 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897e6730 0x55ce89ba7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897de550 0x55ce89bac0e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3313022903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89ba2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3313022903 >> 172.21.15.19:0/3313022903 conn(0x55ce897fb560 msgr2=0x55ce89ba73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3313022903 shutdown_connections 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.971+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3313022903 wait complete. 2024-04-25T10:59:24.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 -- start start 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 Processor -- start 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89bb9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897de550 0x55ce89bba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897e6730 0x55ce89bb5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41ae6c0 con 0x55ce897e6730 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41afb30 con 0x55ce897de550 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41aed90 con 0x55ce897c61f0 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5f1e02740 10 client.0 _ll_register_callbacks cb 0x7fd5e0007f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5e94ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897e6730 0x55ce89bb5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5e8ce9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89bb9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.972+0000 7fd5e94ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897e6730 0x55ce89bb5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53290/0 (socket says 172.21.15.19:53290) 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e94ea640 1 -- 172.21.15.19:0/3708187763 learned_addr learned my addr 172.21.15.19:0/3708187763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897de550 0x55ce89bba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897de550 msgr2=0x55ce89bba400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897de550 0x55ce89bba400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/3708187763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897e6730 msgr2=0x55ce89bb5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897e6730 0x55ce89bb5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e8ce9640 1 -- 172.21.15.19:0/3708187763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce89bb5b80 con 0x55ce897c61f0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e94ea640 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897e6730 0x55ce89bb5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5e8ce9640 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89bb9ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c00 tx=0x7fd5e00afce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd77b3a905a55af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5c3fff640 1 -- 172.21.15.19:0/3708187763 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00a6500 con 0x55ce897c61f0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5c3fff640 1 -- 172.21.15.19:0/3708187763 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd5e00a6200 con 0x55ce897c61f0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d404a070 con 0x55ce897c61f0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5c3fff640 1 -- 172.21.15.19:0/3708187763 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd5e00162d0 con 0x55ce897c61f0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5f1e02740 10 client.25810 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.973+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ce89c2bd50 con 0x55ce897c61f0 2024-04-25T10:59:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5c3fff640 1 -- 172.21.15.19:0/3708187763 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d404a070 con 0x55ce897c61f0 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5c3fff640 1 -- 172.21.15.19:0/3708187763 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd5e00a5a60 con 0x55ce897c61f0 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5f1e02740 10 client.25810 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5f1e02740 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5f1e02740 20 client.25810 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce89807410 0x55ce89c2e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5f1e02740 4 client.25810 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ce89b15580 con 0x55ce89807410 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.974+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce89807410 0x55ce89c2e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.975+0000 7fd5c3fff640 10 client.25810 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.975+0000 7fd5dbfff640 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce89807410 0x55ce89c2e350 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0e0484e65f2804 server_cookie=a17852d33f2c9ccf in_seq=0 out_seq=0 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5c3fff640 1 -- 172.21.15.19:0/3708187763 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ce89b15580 con 0x55ce89807410 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5c3fff640 10 client.25810 handle_command_reply: tid=0 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 client.25810 shutdown 2024-04-25T10:59:24.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 msgr2=0x55ce89bb9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89bb9ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c00 tx=0x7fd5e00afce0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 shutdown_connections 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55ce89807410 0x55ce89c2e350 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce897e6730 0x55ce89bb5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce897de550 0x55ce89bba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 --2- 172.21.15.19:0/3708187763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce897c61f0 0x55ce89bb9ec0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5c3fff640 0 client.25810 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 >> 172.21.15.19:0/3708187763 conn(0x55ce897fb560 msgr2=0x55ce89ba2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 shutdown_connections 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 1 -- 172.21.15.19:0/3708187763 wait complete. 2024-04-25T10:59:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.976+0000 7fd5f1e02740 20 client.25810 trim_cache size 0 max 16384 2024-04-25T10:59:24.982 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:59:24.982 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:59:24.982 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "bede1f55-3706-4506-8d8c-3836168b07f4", 2024-04-25T10:59:24.982 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:59:24.982 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd5cc0afc60 msgr2=0x7fd5cc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd5cc0afc60 0x7fd5cc0b2170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc022000 tx=0x7fd5dc020000 comp rx=0 tx=0).stop 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4103190 msgr2=0x7fd5e419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4103190 0x7fd5e419b660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000d950 tx=0x7fd5e000de10 comp rx=0 tx=0).stop 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 shutdown_connections 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd5cc0afc60 0x7fd5cc0b2170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e4108ed0 0x7fd5e419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5e4108a30 0x7fd5e419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 --2- 172.21.15.19:0/3103580335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e4103190 0x7fd5e419b660 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 >> 172.21.15.19:0/3103580335 conn(0x7fd5e40fe260 msgr2=0x7fd5e40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 shutdown_connections 2024-04-25T10:59:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:24.977+0000 7fd5eaf4d640 1 -- 172.21.15.19:0/3103580335 wait complete. 2024-04-25T10:59:24.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:59:24.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/700173078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a98104c90 msgr2=0x7f9a981050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/700173078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a98104c90 0x7f9a981050a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009900 tx=0x7f9a8802a480 comp rx=0 tx=0).stop 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/700173078 shutdown_connections 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/700173078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810e050 0x7f9a98110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/700173078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9810b6f0 0x7f9a9810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/700173078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a98104c90 0x7f9a981050a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/700173078 >> 172.21.15.19:0/700173078 conn(0x7f9a98078100 msgr2=0x7f9a98078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/700173078 shutdown_connections 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.102+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/700173078 wait complete. 2024-04-25T10:59:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 Processor -- start 2024-04-25T10:59:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 -- start start 2024-04-25T10:59:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a98104c90 0x7f9a9819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810b6f0 0x7f9a9819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a9810e050 0x7f9a9819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98113cb0 con 0x7f9a9810e050 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98113b30 con 0x7f9a9810b6f0 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9f9e3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98113e30 con 0x7f9a98104c90 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a9810e050 0x7f9a9819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a9810e050 0x7f9a9819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53304/0 (socket says 172.21.15.19:53304) 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 -- 172.21.15.19:0/3614789416 learned_addr learned my addr 172.21.15.19:0/3614789416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a98104c90 0x7f9a9819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810b6f0 0x7f9a9819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 -- 172.21.15.19:0/3614789416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a98104c90 msgr2=0x7f9a9819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a98104c90 0x7f9a9819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 -- 172.21.15.19:0/3614789416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810b6f0 msgr2=0x7f9a9819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810b6f0 0x7f9a9819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.103+0000 7f9a9df80640 1 -- 172.21.15.19:0/3614789416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9819f570 con 0x7f9a9810e050 2024-04-25T10:59:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.104+0000 7f9a9df80640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a9810e050 0x7f9a9819ef30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009fd0 tx=0x7f9a9000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.104+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/3614789416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90017070 con 0x7f9a9810e050 2024-04-25T10:59:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.104+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9819f860 con 0x7f9a9810e050 2024-04-25T10:59:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.104+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/3614789416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a90003e20 con 0x7f9a9810e050 2024-04-25T10:59:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.104+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/3614789416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90010460 con 0x7f9a9810e050 2024-04-25T10:59:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.104+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9819fd60 con 0x7f9a9810e050 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 Processor -- start 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/3614789416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9a90003970 con 0x7f9a9810e050 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 -- start start 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a4aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bf39b0d80 con 0x559bf36aa390 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bf396a8b0 con 0x559bf36896b0 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9aa696a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559bf39acd40 con 0x559bf36814d0 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9a9cf7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a4aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9a9cf7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a4aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53314/0 (socket says 172.21.15.19:53314) 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/1808190176 learned_addr learned my addr 172.21.15.19:0/1808190176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.107+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a8e7fc640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9a840afc60 0x7f9a840b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/1808190176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 msgr2=0x559bf3a4afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/1808190176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 msgr2=0x559bf3a578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/1808190176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf393ba40 con 0x559bf36aa390 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a9df80640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9a840afc60 0x7f9a840b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/3614789416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9a900deca0 con 0x7f9a9810e050 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a4aa60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400bd50 tx=0x7f9a940127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ec5f6f212eeba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a837fe640 1 -- 172.21.15.19:0/1808190176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940120d0 con 0x559bf36aa390 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a837fe640 1 -- 172.21.15.19:0/1808190176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a94016070 con 0x559bf36aa390 2024-04-25T10:59:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.108+0000 7f9a837fe640 1 -- 172.21.15.19:0/1808190176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940107a0 con 0x559bf36aa390 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 -- 172.21.15.19:0/1808190176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 msgr2=0x559bf3a4aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9a9df80640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9a840afc60 0x7f9a840b2170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a90005c40 tx=0x7f9a9006d890 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a4aa60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400bd50 tx=0x7f9a940127c0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 -- 172.21.15.19:0/1808190176 shutdown_connections 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 --2- 172.21.15.19:0/1808190176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a4aa60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 -- 172.21.15.19:0/1808190176 >> 172.21.15.19:0/1808190176 conn(0x559bf369e4e0 msgr2=0x559bf3a47210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 -- 172.21.15.19:0/1808190176 shutdown_connections 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 -- 172.21.15.19:0/1808190176 wait complete. 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.109+0000 7f9aa696a740 1 -- start start 2024-04-25T10:59:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 1 Processor -- start 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a56fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36896b0 0x559bf3a57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36aa390 0x559bf3a55460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bf3a57fe0 con 0x559bf36896b0 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bf3937d10 con 0x559bf36aa390 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559bf3a58160 con 0x559bf36814d0 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9aa696a740 10 client.0 _ll_register_callbacks cb 0x559bf3964210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9a9d77f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36896b0 0x559bf3a57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9a9d77f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36896b0 0x559bf3a57500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53322/0 (socket says 172.21.15.19:53322) 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9a9d77f640 1 -- 172.21.15.19:0/4279380668 learned_addr learned my addr 172.21.15.19:0/4279380668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:25.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a56fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.110+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36aa390 0x559bf3a55460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a9d77f640 1 -- 172.21.15.19:0/4279380668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 msgr2=0x559bf3a56fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a56fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a9d77f640 1 -- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36aa390 msgr2=0x559bf3a55460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36aa390 0x559bf3a55460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a9d77f640 1 -- 172.21.15.19:0/4279380668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf378c7d0 con 0x559bf36896b0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36aa390 0x559bf3a55460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36896b0 0x559bf3a57500 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a8802a9a0 tx=0x7f9a880316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdd9b15de6dd4de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a80ff9640 1 -- 172.21.15.19:0/4279380668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88035040 con 0x559bf36896b0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a80ff9640 1 -- 172.21.15.19:0/4279380668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9a88004040 con 0x559bf36896b0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a94016070 con 0x559bf36896b0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a80ff9640 1 -- 172.21.15.19:0/4279380668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9a88032e10 con 0x559bf36896b0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9aa696a740 10 client.16338 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559bf36c23f0 con 0x559bf36896b0 2024-04-25T10:59:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a80ff9640 1 -- 172.21.15.19:0/4279380668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a94016070 con 0x559bf36896b0 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.111+0000 7f9a80ff9640 1 -- 172.21.15.19:0/4279380668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f9a880048e0 con 0x559bf36896b0 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9aa696a740 10 client.16338 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9aa696a740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9aa696a740 20 client.16338 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9aa696a740 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf3668f60 0x559bf3a677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9aa696a740 4 client.16338 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bf39be990 con 0x559bf3668f60 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf3668f60 0x559bf3a677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf3668f60 0x559bf3a677b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3a2617aa511fcc server_cookie=8599e72c5fcaed0 in_seq=0 out_seq=0 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.112+0000 7f9a80ff9640 10 client.16338 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9a80ff9640 1 -- 172.21.15.19:0/4279380668 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559bf39be990 con 0x559bf3668f60 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9a80ff9640 10 client.16338 handle_command_reply: tid=0 2024-04-25T10:59:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 client.16338 shutdown 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36896b0 msgr2=0x559bf3a57500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36896b0 0x559bf3a57500 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a8802a9a0 tx=0x7f9a880316d0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 shutdown_connections 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf3668f60 0x559bf3a677b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36aa390 0x559bf3a55460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36896b0 0x559bf3a57500 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 --2- 172.21.15.19:0/4279380668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a56fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9a80ff9640 0 client.16338 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 >> 172.21.15.19:0/4279380668 conn(0x559bf369e4e0 msgr2=0x559bf3a4c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 shutdown_connections 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 1 -- 172.21.15.19:0/4279380668 wait complete. 2024-04-25T10:59:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.113+0000 7f9aa696a740 20 client.16338 trim_cache size 0 max 16384 2024-04-25T10:59:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.131+0000 7f9aa696a740 1 Processor -- start 2024-04-25T10:59:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9aa696a740 1 -- start start 2024-04-25T10:59:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a53d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a54270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9aa696a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ad820 con 0x559bf36aa390 2024-04-25T10:59:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9aa696a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b00a0 con 0x559bf36896b0 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9aa696a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ade90 con 0x559bf36814d0 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a54270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a54270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53326/0 (socket says 172.21.15.19:53326) 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3867891882 learned_addr learned my addr 172.21.15.19:0/3867891882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a53d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3867891882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 msgr2=0x559bf3a4a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3867891882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 msgr2=0x559bf3a53d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a53d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3867891882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf3a50d20 con 0x559bf36aa390 2024-04-25T10:59:25.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.132+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a54270 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400bd20 tx=0x7f9a940127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a6c0177e815252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9a82ffd640 1 -- 172.21.15.19:0/3867891882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94005110 con 0x559bf36aa390 2024-04-25T10:59:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9a82ffd640 1 -- 172.21.15.19:0/3867891882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a94016040 con 0x559bf36aa390 2024-04-25T10:59:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9a82ffd640 1 -- 172.21.15.19:0/3867891882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9401b6d0 con 0x559bf36aa390 2024-04-25T10:59:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9aa696a740 1 -- 172.21.15.19:0/3867891882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 msgr2=0x559bf3a54270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a54270 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400bd20 tx=0x7f9a940127c0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9aa696a740 1 -- 172.21.15.19:0/3867891882 shutdown_connections 2024-04-25T10:59:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36aa390 0x559bf3a54270 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a53d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3867891882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36814d0 0x559bf3a4a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.133+0000 7f9aa696a740 1 -- 172.21.15.19:0/3867891882 >> 172.21.15.19:0/3867891882 conn(0x559bf369e4e0 msgr2=0x559bf3a46bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 -- 172.21.15.19:0/3867891882 shutdown_connections 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 -- 172.21.15.19:0/3867891882 wait complete. 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 -- start start 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 Processor -- start 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36814d0 0x559bf3a5d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a5aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36aa390 0x559bf3a5aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ae560 con 0x559bf36814d0 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981af9d0 con 0x559bf36896b0 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981aec30 con 0x559bf36aa390 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.134+0000 7f9aa696a740 10 client.0 _ll_register_callbacks cb 0x7f9a88005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36814d0 0x559bf3a5d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36814d0 0x559bf3a5d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53340/0 (socket says 172.21.15.19:53340) 2024-04-25T10:59:25.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3632988616 learned_addr learned my addr 172.21.15.19:0/3632988616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a5aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36aa390 0x559bf3a5aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3632988616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36aa390 msgr2=0x559bf3a5aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36aa390 0x559bf3a5aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 msgr2=0x559bf3a5aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a5aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 -- 172.21.15.19:0/3632988616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf36820f0 con 0x559bf36814d0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a9cf7e640 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36814d0 0x559bf3a5d740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400b8a0 tx=0x7f9a9400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1b360d168a5e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a80ff9640 1 -- 172.21.15.19:0/3632988616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940053e0 con 0x559bf36814d0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a80ff9640 1 -- 172.21.15.19:0/3632988616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9a9402c640 con 0x559bf36814d0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a94016040 con 0x559bf36814d0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9a80ff9640 1 -- 172.21.15.19:0/3632988616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9a9402c840 con 0x559bf36814d0 2024-04-25T10:59:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9aa696a740 10 client.16344 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.135+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559bf39714b0 con 0x559bf36814d0 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9a80ff9640 1 -- 172.21.15.19:0/3632988616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a94016040 con 0x559bf36814d0 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9a80ff9640 1 -- 172.21.15.19:0/3632988616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f9a9401b630 con 0x559bf36814d0 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9aa696a740 10 client.16344 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9aa696a740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9aa696a740 20 client.16344 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf363e3a0 0x559bf3ad1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9aa696a740 4 client.16344 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559bf39b8800 con 0x559bf363e3a0 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.136+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf363e3a0 0x559bf3ad1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9a80ff9640 10 client.16344 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9a9d77f640 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf363e3a0 0x559bf3ad1870 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832e0fae7d5c39e server_cookie=74f0c5b502c68584 in_seq=0 out_seq=0 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9a80ff9640 1 -- 172.21.15.19:0/3632988616 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559bf39b8800 con 0x559bf363e3a0 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9a80ff9640 10 client.16344 handle_command_reply: tid=0 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 client.16344 shutdown 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36814d0 msgr2=0x559bf3a5d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36814d0 0x559bf3a5d740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400b8a0 tx=0x7f9a9400b390 comp rx=0 tx=0).stop 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 shutdown_connections 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559bf363e3a0 0x559bf3ad1870 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bf36aa390 0x559bf3a5aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bf36896b0 0x559bf3a5aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 --2- 172.21.15.19:0/3632988616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bf36814d0 0x559bf3a5d740 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9a80ff9640 0 client.16344 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 >> 172.21.15.19:0/3632988616 conn(0x559bf369e4e0 msgr2=0x559bf3a54cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 shutdown_connections 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 1 -- 172.21.15.19:0/3632988616 wait complete. 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.137+0000 7f9aa696a740 20 client.16344 trim_cache size 0 max 16384 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: "bede1f55-3706-4506-8d8c-3836168b07f4": { 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "bede1f55-3706-4506-8d8c-3836168b07f4", 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:59:25.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:59:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:59:25.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9a840afc60 msgr2=0x7f9a840b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9a840afc60 0x7f9a840b2170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a90005c40 tx=0x7f9a9006d890 comp rx=0 tx=0).stop 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a9810e050 msgr2=0x7f9a9819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a9810e050 0x7f9a9819ef30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009fd0 tx=0x7f9a9000b260 comp rx=0 tx=0).stop 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 shutdown_connections 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9a840afc60 0x7f9a840b2170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a9810e050 0x7f9a9819ef30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810b6f0 0x7f9a9819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 --2- 172.21.15.19:0/3614789416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a98104c90 0x7f9a9819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 >> 172.21.15.19:0/3614789416 conn(0x7f9a98078100 msgr2=0x7f9a98101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 shutdown_connections 2024-04-25T10:59:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:25.138+0000 7f9a9f9e3640 1 -- 172.21.15.19:0/3614789416 wait complete. 2024-04-25T10:59:25.149 INFO:tasks.cephfs.filesystem:scrub status for tag:bede1f55-3706-4506-8d8c-3836168b07f4 - {'path': '/', 'tag': 'bede1f55-3706-4506-8d8c-3836168b07f4', 'options': 'recursive,force'} 2024-04-25T10:59:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:25 smithi019 ceph-mon[29209]: pgmap v150: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:25 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:59:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:25 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:25 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:59:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:25 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:25.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:25 smithi169 ceph-mon[33049]: pgmap v150: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:25.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:25 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:59:25.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:25 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:25.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:25 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:59:25.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:25 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:25.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:25 smithi112 ceph-mon[32799]: pgmap v150: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:25 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:59:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:25 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:25 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:59:25.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:25 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:27.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:27 smithi019 ceph-mon[29209]: pgmap v151: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:27.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:27 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T10:59:27.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:27 smithi169 ceph-mon[33049]: pgmap v151: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:27.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:27 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T10:59:27.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:27 smithi112 ceph-mon[32799]: pgmap v151: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:27.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:27 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T10:59:29.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:29 smithi019 ceph-mon[29209]: pgmap v152: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:29.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:29 smithi169 ceph-mon[33049]: pgmap v152: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:29.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:29 smithi112 ceph-mon[32799]: pgmap v152: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:31.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:31 smithi019 ceph-mon[29209]: pgmap v153: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:31.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:31 smithi169 ceph-mon[33049]: pgmap v153: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:31.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:31 smithi112 ceph-mon[32799]: pgmap v153: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:33.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:33 smithi019 ceph-mon[29209]: pgmap v154: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:33.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:33 smithi169 ceph-mon[33049]: pgmap v154: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:33.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:33 smithi112 ceph-mon[32799]: pgmap v154: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:35.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:35 smithi019 ceph-mon[29209]: pgmap v155: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:35.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:35 smithi169 ceph-mon[33049]: pgmap v155: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:35.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:35 smithi112 ceph-mon[32799]: pgmap v155: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T10:59:37.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:37 smithi019 ceph-mon[29209]: pgmap v156: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:37.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:37 smithi169 ceph-mon[33049]: pgmap v156: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:37.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:37 smithi112 ceph-mon[32799]: pgmap v156: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T10:59:39.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:39 smithi019 ceph-mon[29209]: pgmap v157: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:39 smithi169 ceph-mon[33049]: pgmap v157: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:39.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:39 smithi112 ceph-mon[32799]: pgmap v157: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:40.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:40 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:59:40.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:40 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:59:40.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:40 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:59:41.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:41 smithi169 ceph-mon[33049]: pgmap v158: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:41.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:41 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:59:41.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:41 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:59:41.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:41 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:59:41.710 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:41 smithi112 ceph-mon[32799]: pgmap v158: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:41.710 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:41 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:59:41.710 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:41 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:59:41.710 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:41 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:59:41.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:41 smithi019 ceph-mon[29209]: pgmap v158: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:41 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:59:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:41 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:59:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:41 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T10:59:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:43 smithi169 ceph-mon[33049]: pgmap v159: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:43.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:43 smithi112 ceph-mon[32799]: pgmap v159: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:43.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:43 smithi019 ceph-mon[29209]: pgmap v159: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:45.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:45 smithi169 ceph-mon[33049]: pgmap v160: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:45.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:45 smithi112 ceph-mon[32799]: pgmap v160: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:45.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:45 smithi019 ceph-mon[29209]: pgmap v160: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:47.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:47 smithi169 ceph-mon[33049]: pgmap v161: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:47.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:47 smithi112 ceph-mon[32799]: pgmap v161: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:47.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:47 smithi019 ceph-mon[29209]: pgmap v161: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:49.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:49 smithi169 ceph-mon[33049]: pgmap v162: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:49.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:49 smithi112 ceph-mon[32799]: pgmap v162: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:49.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:49 smithi019 ceph-mon[29209]: pgmap v162: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:51.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:51 smithi019 ceph-mon[29209]: pgmap v163: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:51.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:51 smithi169 ceph-mon[33049]: pgmap v163: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:51.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:51 smithi112 ceph-mon[32799]: pgmap v163: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:53.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:53 smithi169 ceph-mon[33049]: pgmap v164: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:53.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:53 smithi112 ceph-mon[32799]: pgmap v164: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:53.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:53 smithi019 ceph-mon[29209]: pgmap v164: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:55.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:59:55.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:59:55.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.279+0000 7f571b764640 1 -- 172.21.15.19:0/1770622982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 msgr2=0x7f571410f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.279+0000 7f571b764640 1 --2- 172.21.15.19:0/1770622982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f571410f4a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f570c009ca0 tx=0x7f570c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.279+0000 7f571b764640 1 -- 172.21.15.19:0/1770622982 shutdown_connections 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.279+0000 7f571b764640 1 --2- 172.21.15.19:0/1770622982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f571410f4a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.279+0000 7f571b764640 1 --2- 172.21.15.19:0/1770622982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5714108eb0 0x7f5714102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.279+0000 7f571b764640 1 --2- 172.21.15.19:0/1770622982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5714108a10 0x7f5714102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.280+0000 7f571b764640 1 -- 172.21.15.19:0/1770622982 >> 172.21.15.19:0/1770622982 conn(0x7f5714078110 msgr2=0x7f5714078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.280+0000 7f571b764640 1 -- 172.21.15.19:0/1770622982 shutdown_connections 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.280+0000 7f571b764640 1 -- 172.21.15.19:0/1770622982 wait complete. 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.280+0000 7f571b764640 1 Processor -- start 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.280+0000 7f571b764640 1 -- start start 2024-04-25T10:59:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f571b764640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f57141a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f571b764640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5714108a10 0x7f57141a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f571b764640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5714108eb0 0x7f571419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f571b764640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112c30 con 0x7f57141031c0 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f571b764640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112ab0 con 0x7f5714108a10 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f571b764640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112db0 con 0x7f5714108eb0 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f57141a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f57141a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47988/0 (socket says 172.21.15.19:47988) 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 -- 172.21.15.19:0/4245943110 learned_addr learned my addr 172.21.15.19:0/4245943110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5718cff640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5714108a10 0x7f57141a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719d01640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5714108eb0 0x7f571419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 -- 172.21.15.19:0/4245943110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5714108eb0 msgr2=0x7f571419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5714108eb0 0x7f571419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 -- 172.21.15.19:0/4245943110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5714108a10 msgr2=0x7f57141a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5714108a10 0x7f57141a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 -- 172.21.15.19:0/4245943110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571419cd60 con 0x7f57141031c0 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719d01640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5714108eb0 0x7f571419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f5719500640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f57141a25a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f571000d740 tx=0x7f571000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.281+0000 7f570a7fc640 1 -- 172.21.15.19:0/4245943110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710017070 con 0x7f57141031c0 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.282+0000 7f570a7fc640 1 -- 172.21.15.19:0/4245943110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f571000de50 con 0x7f57141031c0 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.282+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571419d050 con 0x7f57141031c0 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.282+0000 7f570a7fc640 1 -- 172.21.15.19:0/4245943110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571000f620 con 0x7f57141031c0 2024-04-25T10:59:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.282+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141a9340 con 0x7f57141031c0 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.284+0000 7f570a7fc640 1 -- 172.21.15.19:0/4245943110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f571001a070 con 0x7f57141031c0 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 Processor -- start 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 -- start start 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f44390 0x55a7a92e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a7a924ad80 con 0x55a7a8f44390 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7a92048b0 con 0x55a7a8f236b0 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5722612740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a7a9246d40 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f44390 0x55a7a92e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f44390 0x55a7a92e4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47992/0 (socket says 172.21.15.19:47992) 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 -- 172.21.15.19:0/1508086026 learned_addr learned my addr 172.21.15.19:0/1508086026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5718cff640 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719500640 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f570a7fc640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f57000afce0 0x7f57000b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 -- 172.21.15.19:0/1508086026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f1b4d0 msgr2=0x55a7a92e4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92e4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 -- 172.21.15.19:0/1508086026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 msgr2=0x55a7a92f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.285+0000 7f5719d01640 1 -- 172.21.15.19:0/1508086026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a91d5a40 con 0x55a7a8f44390 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.286+0000 7f5719d01640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f57000afce0 0x7f57000b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.286+0000 7f570a7fc640 1 -- 172.21.15.19:0/4245943110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f57100dea60 con 0x7f57141031c0 2024-04-25T10:59:55.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.286+0000 7f5719d01640 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f44390 0x55a7a92e4a60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f570c006010 tx=0x7f570c009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5a92d27f5285e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.286+0000 7f56ff7fe640 1 -- 172.21.15.19:0/1508086026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c003e40 con 0x55a7a8f44390 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.286+0000 7f56ff7fe640 1 -- 172.21.15.19:0/1508086026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c036070 con 0x55a7a8f44390 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.286+0000 7f5719d01640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f57000afce0 0x7f57000b21f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f570c04b000 tx=0x7f570c048000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.286+0000 7f56ff7fe640 1 -- 172.21.15.19:0/1508086026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c02fa50 con 0x55a7a8f44390 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 -- 172.21.15.19:0/1508086026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f44390 msgr2=0x55a7a92e4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f44390 0x55a7a92e4a60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f570c006010 tx=0x7f570c009a00 comp rx=0 tx=0).stop 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 -- 172.21.15.19:0/1508086026 shutdown_connections 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 --2- 172.21.15.19:0/1508086026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f44390 0x55a7a92e4a60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 -- 172.21.15.19:0/1508086026 >> 172.21.15.19:0/1508086026 conn(0x55a7a8f384e0 msgr2=0x55a7a92e1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 -- 172.21.15.19:0/1508086026 shutdown_connections 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 -- 172.21.15.19:0/1508086026 wait complete. 2024-04-25T10:59:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 -- start start 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.287+0000 7f5722612740 1 Processor -- start 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5722612740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5722612740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5722612740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a92f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5722612740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a7a92f1e90 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5722612740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7a91d1d10 con 0x55a7a8f236b0 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5722612740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a7a92f2010 con 0x55a7a8f44390 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5722612740 10 client.0 _ll_register_callbacks cb 0x55a7a91fe210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47994/0 (socket says 172.21.15.19:47994) 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 -- 172.21.15.19:0/2758056208 learned_addr learned my addr 172.21.15.19:0/2758056208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5718cff640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719d01640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a92f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 -- 172.21.15.19:0/2758056208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 msgr2=0x55a7a92f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a92f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 -- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 msgr2=0x55a7a92efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719500640 1 -- 172.21.15.19:0/2758056208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a90267d0 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.288+0000 7f5719d01640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a92f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5719500640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ef600 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f57100dc240 tx=0x7f5710010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aafdc575496e53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f56fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f56fcff9640 1 -- 172.21.15.19:0/2758056208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710021070 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f56fcff9640 1 -- 172.21.15.19:0/2758056208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f57100a5e90 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570c036070 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f56fcff9640 1 -- 172.21.15.19:0/2758056208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f57100a63a0 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 10 client.16356 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a7a8f5c3f0 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f56fcff9640 1 -- 172.21.15.19:0/2758056208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c036070 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f56fcff9640 1 -- 172.21.15.19:0/2758056208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f57100a5900 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 10 client.16356 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 20 client.16356 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8f02f60 0x55a7a93014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 4 client.16356 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.289+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7a9258990 con 0x55a7a8f02f60 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.290+0000 7f5718cff640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8f02f60 0x55a7a93014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.290+0000 7f5718cff640 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8f02f60 0x55a7a93014f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cdb38a2c55b906f server_cookie=e377c4f30a6b02b4 in_seq=0 out_seq=0 2024-04-25T10:59:55.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.290+0000 7f56fcff9640 10 client.16356 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f56fcff9640 1 -- 172.21.15.19:0/2758056208 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7a9258990 con 0x55a7a8f02f60 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f56fcff9640 10 client.16356 handle_command_reply: tid=0 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 client.16356 shutdown 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 msgr2=0x55a7a92ef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ef600 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f57100dc240 tx=0x7f5710010c70 comp rx=0 tx=0).stop 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 shutdown_connections 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8f02f60 0x55a7a93014f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a92f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f236b0 0x55a7a92efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f56fcff9640 0 client.16356 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 --2- 172.21.15.19:0/2758056208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ef600 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 >> 172.21.15.19:0/2758056208 conn(0x55a7a8f384e0 msgr2=0x55a7a92e6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 shutdown_connections 2024-04-25T10:59:55.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.291+0000 7f5722612740 1 -- 172.21.15.19:0/2758056208 wait complete. 2024-04-25T10:59:55.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.292+0000 7f5722612740 20 client.16356 trim_cache size 0 max 16384 2024-04-25T10:59:55.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 Processor -- start 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 -- start start 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f236b0 0x55a7a92ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f44390 0x55a7a92f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57141af9f0 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b2270 con 0x55a7a8f44390 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.310+0000 7f5722612740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b0060 con 0x55a7a8f236b0 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ee6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48002/0 (socket says 172.21.15.19:48002) 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 -- 172.21.15.19:0/3506860141 learned_addr learned my addr 172.21.15.19:0/3506860141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5718cff640 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f236b0 0x55a7a92ea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 -- 172.21.15.19:0/3506860141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f236b0 msgr2=0x55a7a92ea5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f236b0 0x55a7a92ea5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 -- 172.21.15.19:0/3506860141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f44390 msgr2=0x55a7a92f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f44390 0x55a7a92f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 -- 172.21.15.19:0/3506860141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a8f1c0f0 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f5719500640 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ee6f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f57100a2cb0 tx=0x7f5710000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ee4149d04b2431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f56feffd640 1 -- 172.21.15.19:0/3506860141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100c5030 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f56feffd640 1 -- 172.21.15.19:0/3506860141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57100a6740 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.311+0000 7f56feffd640 1 -- 172.21.15.19:0/3506860141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710021040 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 -- 172.21.15.19:0/3506860141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 msgr2=0x55a7a92ee6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ee6f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f57100a2cb0 tx=0x7f5710000bc0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 -- 172.21.15.19:0/3506860141 shutdown_connections 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f44390 0x55a7a92f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f236b0 0x55a7a92ea5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 --2- 172.21.15.19:0/3506860141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92ee6f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 -- 172.21.15.19:0/3506860141 >> 172.21.15.19:0/3506860141 conn(0x55a7a8f384e0 msgr2=0x55a7a92e6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 -- 172.21.15.19:0/3506860141 shutdown_connections 2024-04-25T10:59:55.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 -- 172.21.15.19:0/3506860141 wait complete. 2024-04-25T10:59:55.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.312+0000 7f5722612740 1 -- start start 2024-04-25T10:59:55.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 1 Processor -- start 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f236b0 0x55a7a92f7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a9361820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b0730 con 0x55a7a8f236b0 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b1ba0 con 0x55a7a8f1b4d0 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b0e00 con 0x55a7a8f44390 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5722612740 10 client.0 _ll_register_callbacks cb 0x7f5710009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f236b0 0x55a7a92f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f236b0 0x55a7a92f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48018/0 (socket says 172.21.15.19:48018) 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 -- 172.21.15.19:0/857156982 learned_addr learned my addr 172.21.15.19:0/857156982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5719d01640 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a9361820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5719500640 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 -- 172.21.15.19:0/857156982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 msgr2=0x55a7a9361820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a9361820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 -- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f1b4d0 msgr2=0x55a7a92f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.313+0000 7f5718cff640 1 -- 172.21.15.19:0/857156982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a920b4b0 con 0x55a7a8f236b0 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f5718cff640 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f236b0 0x55a7a92f7060 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5704004700 tx=0x7f57040039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80c6b7f6e9c0aec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f56fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f56fcff9640 1 -- 172.21.15.19:0/857156982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570401c880 con 0x55a7a8f236b0 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f56fcff9640 1 -- 172.21.15.19:0/857156982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f570401ca20 con 0x55a7a8f236b0 2024-04-25T10:59:55.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57100a6740 con 0x55a7a8f236b0 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f56fcff9640 1 -- 172.21.15.19:0/857156982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f570402a050 con 0x55a7a8f236b0 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f5722612740 10 client.16362 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a7a9368fb0 con 0x55a7a8f236b0 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f56fcff9640 1 -- 172.21.15.19:0/857156982 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57100a6740 con 0x55a7a8f236b0 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f56fcff9640 1 -- 172.21.15.19:0/857156982 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f57040255a0 con 0x55a7a8f236b0 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f5722612740 10 client.16362 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f5722612740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.314+0000 7f5722612740 20 client.16362 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.315+0000 7f5722612740 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8ed83a0 0x55a7a936b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.315+0000 7f5722612740 4 client.16362 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.315+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a7a9252800 con 0x55a7a8ed83a0 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.315+0000 7f5719500640 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8ed83a0 0x55a7a936b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.315+0000 7f56fcff9640 10 client.16362 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.315+0000 7f5719500640 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8ed83a0 0x55a7a936b7a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2defb45cf20bcd server_cookie=7260a8c3064e4eef in_seq=0 out_seq=0 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f56fcff9640 1 -- 172.21.15.19:0/857156982 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a7a9252800 con 0x55a7a8ed83a0 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f56fcff9640 10 client.16362 handle_command_reply: tid=0 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 client.16362 shutdown 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f236b0 msgr2=0x55a7a92f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f236b0 0x55a7a92f7060 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5704004700 tx=0x7f57040039c0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 shutdown_connections 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a7a8ed83a0 0x55a7a936b7a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a7a8f44390 0x55a7a9361820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7a8f236b0 0x55a7a92f7060 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f56fcff9640 0 client.16362 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 --2- 172.21.15.19:0/857156982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7a8f1b4d0 0x55a7a92f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 >> 172.21.15.19:0/857156982 conn(0x55a7a8f384e0 msgr2=0x55a7a92e6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 shutdown_connections 2024-04-25T10:59:55.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 1 -- 172.21.15.19:0/857156982 wait complete. 2024-04-25T10:59:55.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.316+0000 7f5722612740 20 client.16362 trim_cache size 0 max 16384 2024-04-25T10:59:55.323 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:59:55.323 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:59:55.323 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:59:55.323 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:59:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f57000afce0 msgr2=0x7f57000b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f57000afce0 0x7f57000b21f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f570c04b000 tx=0x7f570c048000 comp rx=0 tx=0).stop 2024-04-25T10:59:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 msgr2=0x7f57141a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f57141a25a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f571000d740 tx=0x7f571000dc00 comp rx=0 tx=0).stop 2024-04-25T10:59:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 shutdown_connections 2024-04-25T10:59:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f57000afce0 0x7f57000b21f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5714108eb0 0x7f571419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5714108a10 0x7f57141a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 --2- 172.21.15.19:0/4245943110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57141031c0 0x7f57141a25a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 >> 172.21.15.19:0/4245943110 conn(0x7f5714078110 msgr2=0x7f5714074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 shutdown_connections 2024-04-25T10:59:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.317+0000 7f571b764640 1 -- 172.21.15.19:0/4245943110 wait complete. 2024-04-25T10:59:55.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:59:55.329 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:55.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.440+0000 7f87f224c640 1 -- 172.21.15.19:0/2669104200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec109fd0 msgr2=0x7f87ec103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.440+0000 7f87f224c640 1 --2- 172.21.15.19:0/2669104200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec109fd0 0x7f87ec103c10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009900 tx=0x7f87d402a410 comp rx=0 tx=0).stop 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 -- 172.21.15.19:0/2669104200 shutdown_connections 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 --2- 172.21.15.19:0/2669104200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87ec104280 0x7f87ec110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 --2- 172.21.15.19:0/2669104200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec109fd0 0x7f87ec103c10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 --2- 172.21.15.19:0/2669104200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87ec109b30 0x7f87ec1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 -- 172.21.15.19:0/2669104200 >> 172.21.15.19:0/2669104200 conn(0x7f87ec0ff210 msgr2=0x7f87ec101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 -- 172.21.15.19:0/2669104200 shutdown_connections 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 -- 172.21.15.19:0/2669104200 wait complete. 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 Processor -- start 2024-04-25T10:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.441+0000 7f87f224c640 1 -- start start 2024-04-25T10:59:55.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87f224c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec104280 0x7f87ec198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87f224c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87ec109b30 0x7f87ec198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87f224c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87ec109fd0 0x7f87ec19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87f224c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec111b60 con 0x7f87ec104280 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87f224c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1119e0 con 0x7f87ec109fd0 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87f224c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec111ce0 con 0x7f87ec109b30 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec104280 0x7f87ec198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec104280 0x7f87ec198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48034/0 (socket says 172.21.15.19:48034) 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 -- 172.21.15.19:0/3225318073 learned_addr learned my addr 172.21.15.19:0/3225318073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87ebfff640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87ec109fd0 0x7f87ec19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eaffd640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87ec109b30 0x7f87ec198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 -- 172.21.15.19:0/3225318073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87ec109b30 msgr2=0x7f87ec198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87ec109b30 0x7f87ec198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 -- 172.21.15.19:0/3225318073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87ec109fd0 msgr2=0x7f87ec19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87ec109fd0 0x7f87ec19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 -- 172.21.15.19:0/3225318073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec19d3c0 con 0x7f87ec104280 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87ebfff640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87ec109fd0 0x7f87ec19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87eb7fe640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec104280 0x7f87ec198560 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00eb90 tx=0x7f87dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87e8ff9640 1 -- 172.21.15.19:0/3225318073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc010630 con 0x7f87ec104280 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec19d6b0 con 0x7f87ec104280 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.442+0000 7f87e8ff9640 1 -- 172.21.15.19:0/3225318073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87dc00ee90 con 0x7f87ec104280 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.443+0000 7f87e8ff9640 1 -- 172.21.15.19:0/3225318073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc018ac0 con 0x7f87ec104280 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.443+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec1a5030 con 0x7f87ec104280 2024-04-25T10:59:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.444+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ec06a190 con 0x7f87ec104280 2024-04-25T10:59:55.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.445+0000 7f87e8ff9640 1 -- 172.21.15.19:0/3225318073 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f87dc018c60 con 0x7f87ec104280 2024-04-25T10:59:55.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.445+0000 7f87e8ff9640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87d80afc60 0x7f87d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.446+0000 7f87eaffd640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87d80afc60 0x7f87d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.446+0000 7f87e8ff9640 1 -- 172.21.15.19:0/3225318073 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f87dc0dfbd0 con 0x7f87ec104280 2024-04-25T10:59:55.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.446+0000 7f87eaffd640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87d80afc60 0x7f87d80b2170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87ec10ea20 tx=0x7f87d4005fd0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.449+0000 7f87e8ff9640 1 -- 172.21.15.19:0/3225318073 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87dc0a8440 con 0x7f87ec104280 2024-04-25T10:59:55.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:55 smithi019 ceph-mon[29209]: pgmap v165: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.674+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87ec1a5200 con 0x7f87ec104280 2024-04-25T10:59:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.674+0000 7f87e8ff9640 1 -- 172.21.15.19:0/3225318073 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f87dc0a7d40 con 0x7f87ec104280 2024-04-25T10:59:55.675 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:59:55.675 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T10:59:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87d80afc60 msgr2=0x7f87d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87d80afc60 0x7f87d80b2170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87ec10ea20 tx=0x7f87d4005fd0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec104280 msgr2=0x7f87ec198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec104280 0x7f87ec198560 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00eb90 tx=0x7f87dc00c6a0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 shutdown_connections 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87d80afc60 0x7f87d80b2170 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87ec109fd0 0x7f87ec19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87ec109b30 0x7f87ec198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 --2- 172.21.15.19:0/3225318073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87ec104280 0x7f87ec198560 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.676+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 >> 172.21.15.19:0/3225318073 conn(0x7f87ec0ff210 msgr2=0x7f87ec10f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.677+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 shutdown_connections 2024-04-25T10:59:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.677+0000 7f87f224c640 1 -- 172.21.15.19:0/3225318073 wait complete. 2024-04-25T10:59:55.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:59:55.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:55 smithi169 ceph-mon[33049]: pgmap v165: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 -- 172.21.15.19:0/3707414961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 msgr2=0x7f3108110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 --2- 172.21.15.19:0/3707414961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108110630 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009ca0 tx=0x7f30f802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 -- 172.21.15.19:0/3707414961 shutdown_connections 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 --2- 172.21.15.19:0/3707414961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108110630 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 --2- 172.21.15.19:0/3707414961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3108109fb0 0x7f3108103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 --2- 172.21.15.19:0/3707414961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3108109b10 0x7f31081036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 -- 172.21.15.19:0/3707414961 >> 172.21.15.19:0/3707414961 conn(0x7f31080ff210 msgr2=0x7f3108101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 -- 172.21.15.19:0/3707414961 shutdown_connections 2024-04-25T10:59:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.803+0000 7f310f900640 1 -- 172.21.15.19:0/3707414961 wait complete. 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 Processor -- start 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 -- start start 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3108109b10 0x7f31080764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3108109fb0 0x7f3108076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3108113d00 con 0x7f3108104260 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3108113b80 con 0x7f3108109fb0 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310f900640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3108113e80 con 0x7f3108109b10 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310d69c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310d69c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48058/0 (socket says 172.21.15.19:48058) 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.804+0000 7f310d69c640 1 -- 172.21.15.19:0/3877137859 learned_addr learned my addr 172.21.15.19:0/3877137859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310ce9b640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3108109b10 0x7f31080764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310de9d640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3108109fb0 0x7f3108076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310d69c640 1 -- 172.21.15.19:0/3877137859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3108109b10 msgr2=0x7f31080764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310d69c640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3108109b10 0x7f31080764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310d69c640 1 -- 172.21.15.19:0/3877137859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3108109fb0 msgr2=0x7f3108076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310d69c640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3108109fb0 0x7f3108076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310d69c640 1 -- 172.21.15.19:0/3877137859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31081a8a40 con 0x7f3108104260 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310de9d640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3108109fb0 0x7f3108076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310d69c640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108075fb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f310000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f31067fc640 1 -- 172.21.15.19:0/3877137859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100017070 con 0x7f3108104260 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31081a8d30 con 0x7f3108104260 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f31067fc640 1 -- 172.21.15.19:0/3877137859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3100009c30 con 0x7f3108104260 2024-04-25T10:59:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f31067fc640 1 -- 172.21.15.19:0/3877137859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100003d30 con 0x7f3108104260 2024-04-25T10:59:55.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.805+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31081a9210 con 0x7f3108104260 2024-04-25T10:59:55.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:55 smithi112 ceph-mon[32799]: pgmap v165: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 Processor -- start 2024-04-25T10:59:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 -- start start 2024-04-25T10:59:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31067fc640 1 -- 172.21.15.19:0/3877137859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3100003ed0 con 0x7f3108104260 2024-04-25T10:59:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cf5390 0x558033096810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x558033096d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558032ffbd80 con 0x558032cd46b0 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558032fb5a20 con 0x558032ccc4d0 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f31167b2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558032ff7d40 con 0x558032cf5390 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f310ce9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f310ce9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48062/0 (socket says 172.21.15.19:48062) 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f310ce9b640 1 -- 172.21.15.19:0/3164783286 learned_addr learned my addr 172.21.15.19:0/3164783286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.808+0000 7f310d69c640 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x558033096d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310de9d640 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cf5390 0x558033096810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310ce9b640 1 -- 172.21.15.19:0/3164783286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cf5390 msgr2=0x558033096810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310ce9b640 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cf5390 0x558033096810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310ce9b640 1 -- 172.21.15.19:0/3164783286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 msgr2=0x558033096d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310ce9b640 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x558033096d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310ce9b640 1 -- 172.21.15.19:0/3164783286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558032fc0c30 con 0x558032cd46b0 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310ce9b640 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a36a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009b30 tx=0x7f30fc0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b5efe1a85869c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f31067fc640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30f00afc60 0x7f30f00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f30ef7fe640 1 -- 172.21.15.19:0/3164783286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc011e10 con 0x558032cd46b0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f310de9d640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30f00afc60 0x7f30f00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f30ef7fe640 1 -- 172.21.15.19:0/3164783286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc016070 con 0x558032cd46b0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f31067fc640 1 -- 172.21.15.19:0/3877137859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f31000ded50 con 0x7f3108104260 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.809+0000 7f30ef7fe640 1 -- 172.21.15.19:0/3164783286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc004a00 con 0x558032cd46b0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 -- 172.21.15.19:0/3164783286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 msgr2=0x5580330a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f310de9d640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30f00afc60 0x7f30f00b2170 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x558033093980 tx=0x7f30f8005d80 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a36a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009b30 tx=0x7f30fc0113e0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 -- 172.21.15.19:0/3164783286 shutdown_connections 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a36a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x558033096d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 --2- 172.21.15.19:0/3164783286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cf5390 0x558033096810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 -- 172.21.15.19:0/3164783286 >> 172.21.15.19:0/3164783286 conn(0x558032ce94e0 msgr2=0x558033092fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 -- 172.21.15.19:0/3164783286 shutdown_connections 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.810+0000 7f31167b2740 1 -- 172.21.15.19:0/3164783286 wait complete. 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 -- start start 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 Processor -- start 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032ccc4d0 0x5580330a1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032cd46b0 0x5580330a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cf5390 0x5580330ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5580330a3d90 con 0x558032cf5390 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558032f82dc0 con 0x558032cd46b0 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580330a3f10 con 0x558032ccc4d0 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f31167b2740 10 client.0 _ll_register_callbacks cb 0x558032f9e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f310de9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cf5390 0x5580330ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f310de9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cf5390 0x5580330ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48068/0 (socket says 172.21.15.19:48068) 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f310de9d640 1 -- 172.21.15.19:0/1233513543 learned_addr learned my addr 172.21.15.19:0/1233513543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.811+0000 7f310ce9b640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032cd46b0 0x5580330a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310d69c640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032ccc4d0 0x5580330a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310de9d640 1 -- 172.21.15.19:0/1233513543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032ccc4d0 msgr2=0x5580330a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310de9d640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032ccc4d0 0x5580330a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310de9d640 1 -- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032cd46b0 msgr2=0x5580330a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310de9d640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032cd46b0 0x5580330a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310de9d640 1 -- 172.21.15.19:0/1233513543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558032d0d3f0 con 0x558032cf5390 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310d69c640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032ccc4d0 0x5580330a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f310de9d640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cf5390 0x5580330ad210 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f30f8035000 tx=0x7f30f802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf1329aea9fa9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f30ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f30ecff9640 1 -- 172.21.15.19:0/1233513543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80318e0 con 0x558032cf5390 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30fc016070 con 0x558032cf5390 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f30ecff9640 1 -- 172.21.15.19:0/1233513543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f30f8004250 con 0x558032cf5390 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f30ecff9640 1 -- 172.21.15.19:0/1233513543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f30f8004450 con 0x558032cf5390 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f31167b2740 10 client.16380 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558032dd77d0 con 0x558032cf5390 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.812+0000 7f30ecff9640 1 -- 172.21.15.19:0/1233513543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc016070 con 0x558032cf5390 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f30ecff9640 1 -- 172.21.15.19:0/1233513543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f30f8050070 con 0x558032cf5390 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f31167b2740 10 client.16380 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f31167b2740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f31167b2740 20 client.16380 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f31167b2740 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cb3f60 0x5580330b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f31167b2740 4 client.16380 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558033009990 con 0x558032cb3f60 2024-04-25T10:59:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f310d69c640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cb3f60 0x5580330b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f30ecff9640 10 client.16380 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.813+0000 7f310d69c640 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cb3f60 0x5580330b3680 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc87770111119da2 server_cookie=70eee09fe2008a56 in_seq=0 out_seq=0 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f30ecff9640 1 -- 172.21.15.19:0/1233513543 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558033009990 con 0x558032cb3f60 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f30ecff9640 10 client.16380 handle_command_reply: tid=0 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 client.16380 shutdown 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cf5390 msgr2=0x5580330ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cf5390 0x5580330ad210 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f30f8035000 tx=0x7f30f802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 shutdown_connections 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cb3f60 0x5580330b3680 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cf5390 0x5580330ad210 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f30ecff9640 0 client.16380 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032cd46b0 0x5580330a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 --2- 172.21.15.19:0/1233513543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032ccc4d0 0x5580330a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.814+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 >> 172.21.15.19:0/1233513543 conn(0x558032ce94e0 msgr2=0x558033098530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.815+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 shutdown_connections 2024-04-25T10:59:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.815+0000 7f31167b2740 1 -- 172.21.15.19:0/1233513543 wait complete. 2024-04-25T10:59:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.815+0000 7f31167b2740 20 client.16380 trim_cache size 0 max 16384 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.828+0000 7f31167b2740 1 Processor -- start 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.828+0000 7f31167b2740 1 -- start start 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f31167b2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f31167b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f31167b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f31167b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31081afae0 con 0x558032cd46b0 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f31167b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b2360 con 0x558032ccc4d0 2024-04-25T10:59:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f31167b2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b0150 con 0x558032cb3f60 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48078/0 (socket says 172.21.15.19:48078) 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 -- 172.21.15.19:0/719077718 learned_addr learned my addr 172.21.15.19:0/719077718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310d69c640 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310ce9b640 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 -- 172.21.15.19:0/719077718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 msgr2=0x5580330ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 -- 172.21.15.19:0/719077718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 msgr2=0x5580330b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 -- 172.21.15.19:0/719077718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580330a1f80 con 0x558032cd46b0 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310ce9b640 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f310de9d640 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330b9380 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f30f8003fd0 tx=0x7f30f8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf03fc66418648f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f30eeffd640 1 -- 172.21.15.19:0/719077718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f805f020 con 0x558032cd46b0 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.829+0000 7f30eeffd640 1 -- 172.21.15.19:0/719077718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f8036050 con 0x558032cd46b0 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f30eeffd640 1 -- 172.21.15.19:0/719077718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f802f8c0 con 0x558032cd46b0 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 -- 172.21.15.19:0/719077718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 msgr2=0x5580330b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330b9380 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f30f8003fd0 tx=0x7f30f8032f20 comp rx=0 tx=0).stop 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 -- 172.21.15.19:0/719077718 shutdown_connections 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330b9380 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 --2- 172.21.15.19:0/719077718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 -- 172.21.15.19:0/719077718 >> 172.21.15.19:0/719077718 conn(0x558032ce94e0 msgr2=0x55803309f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 -- 172.21.15.19:0/719077718 shutdown_connections 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.830+0000 7f31167b2740 1 -- 172.21.15.19:0/719077718 wait complete. 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 -- start start 2024-04-25T10:59:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 Processor -- start 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b0820 con 0x558032cd46b0 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b1c90 con 0x558032ccc4d0 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b0ef0 con 0x558032cb3f60 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f31167b2740 10 client.0 _ll_register_callbacks cb 0x7f30f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f310de9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f310de9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48088/0 (socket says 172.21.15.19:48088) 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f310de9d640 1 -- 172.21.15.19:0/1032941143 learned_addr learned my addr 172.21.15.19:0/1032941143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f310d69c640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.831+0000 7f310ce9b640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f310de9d640 1 -- 172.21.15.19:0/1032941143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 msgr2=0x5580330a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f310de9d640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f310de9d640 1 -- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 msgr2=0x5580330a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f310de9d640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f310de9d640 1 -- 172.21.15.19:0/1032941143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580330a5590 con 0x558032cd46b0 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f310ce9b640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f310de9d640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a4f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005e30 tx=0x7f30f8004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83194a00562fb604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f30ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:55.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f30ecff9640 1 -- 172.21.15.19:0/1032941143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f805f040 con 0x558032cd46b0 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f30ecff9640 1 -- 172.21.15.19:0/1032941143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f30f8030e70 con 0x558032cd46b0 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f8036050 con 0x558032cd46b0 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f30ecff9640 1 -- 172.21.15.19:0/1032941143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f30f8003d40 con 0x558032cd46b0 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f31167b2740 10 client.16386 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55803311b760 con 0x558032cd46b0 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f30ecff9640 1 -- 172.21.15.19:0/1032941143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f8036050 con 0x558032cd46b0 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.832+0000 7f30ecff9640 1 -- 172.21.15.19:0/1032941143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f30f8050070 con 0x558032cd46b0 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f31167b2740 10 client.16386 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f31167b2740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f31167b2740 20 client.16386 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f31167b2740 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cf5390 0x55803311ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f31167b2740 4 client.16386 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:59:55.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558033003800 con 0x558032cf5390 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f310d69c640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cf5390 0x55803311ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f310d69c640 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cf5390 0x55803311ddb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a0f7661f210d52 server_cookie=d2e84e37d730ad04 in_seq=0 out_seq=0 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.833+0000 7f30ecff9640 10 client.16386 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f30ecff9640 1 -- 172.21.15.19:0/1032941143 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558033003800 con 0x558032cf5390 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f30ecff9640 10 client.16386 handle_command_reply: tid=0 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 client.16386 shutdown 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 msgr2=0x5580330a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a4f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005e30 tx=0x7f30f8004880 comp rx=0 tx=0).stop 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 shutdown_connections 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558032cf5390 0x55803311ddb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558032cd46b0 0x5580330a4f20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558032ccc4d0 0x5580330a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 --2- 172.21.15.19:0/1032941143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558032cb3f60 0x5580330a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f30ecff9640 0 client.16386 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:55.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 >> 172.21.15.19:0/1032941143 conn(0x558032ce94e0 msgr2=0x5580330965a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 shutdown_connections 2024-04-25T10:59:55.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 1 -- 172.21.15.19:0/1032941143 wait complete. 2024-04-25T10:59:55.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f31167b2740 20 client.16386 trim_cache size 0 max 16384 2024-04-25T10:59:55.841 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30f00afc60 msgr2=0x7f30f00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.834+0000 7f310f900640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30f00afc60 0x7f30f00b2170 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x558033093980 tx=0x7f30f8005d80 comp rx=0 tx=0).stop 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 msgr2=0x7f3108075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108075fb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f310000dde0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 shutdown_connections 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f30f00afc60 0x7f30f00b2170 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3108109fb0 0x7f3108076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3108109b10 0x7f31080764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 --2- 172.21.15.19:0/3877137859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3108104260 0x7f3108075fb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 >> 172.21.15.19:0/3877137859 conn(0x7f31080ff210 msgr2=0x7f3108106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 shutdown_connections 2024-04-25T10:59:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:55.835+0000 7f310f900640 1 -- 172.21.15.19:0/3877137859 wait complete. 2024-04-25T10:59:56.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:56 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3225318073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:59:56.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:56 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3225318073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:59:56.850 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:59:56.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:56.850 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:59:56.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:56 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3225318073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:59:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 -- 172.21.15.19:0/3138397129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 msgr2=0x7fccd410f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 --2- 172.21.15.19:0/3138397129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd410f7b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009ca0 tx=0x7fcccc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 -- 172.21.15.19:0/3138397129 shutdown_connections 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 --2- 172.21.15.19:0/3138397129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd410f7b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 --2- 172.21.15.19:0/3138397129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4102b00 0x7fccd4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 --2- 172.21.15.19:0/3138397129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4108a30 0x7fccd41025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 -- 172.21.15.19:0/3138397129 >> 172.21.15.19:0/3138397129 conn(0x7fccd40fe260 msgr2=0x7fccd4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 -- 172.21.15.19:0/3138397129 shutdown_connections 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.980+0000 7fccda4f0640 1 -- 172.21.15.19:0/3138397129 wait complete. 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.981+0000 7fccda4f0640 1 Processor -- start 2024-04-25T10:59:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.981+0000 7fccda4f0640 1 -- start start 2024-04-25T10:59:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccda4f0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102b00 0x7fccd419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccda4f0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4108a30 0x7fccd419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccda4f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccda4f0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccd4112c10 con 0x7fccd410d390 2024-04-25T10:59:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccda4f0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fccd4112a90 con 0x7fccd4102b00 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccda4f0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccd4112d90 con 0x7fccd4108a30 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48100/0 (socket says 172.21.15.19:48100) 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 -- 172.21.15.19:0/2840694837 learned_addr learned my addr 172.21.15.19:0/2840694837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd3fff640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102b00 0x7fccd419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd37fe640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4108a30 0x7fccd419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 -- 172.21.15.19:0/2840694837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4108a30 msgr2=0x7fccd419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4108a30 0x7fccd419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 -- 172.21.15.19:0/2840694837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102b00 msgr2=0x7fccd419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102b00 0x7fccd419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 -- 172.21.15.19:0/2840694837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd419f500 con 0x7fccd410d390 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd3fff640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102b00 0x7fccd419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd8a8d640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd419eec0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005cc0 tx=0x7fcccc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd17fa640 1 -- 172.21.15.19:0/2840694837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc003bf0 con 0x7fccd410d390 2024-04-25T10:59:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd17fa640 1 -- 172.21.15.19:0/2840694837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcccc003d90 con 0x7fccd410d390 2024-04-25T10:59:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd419f790 con 0x7fccd410d390 2024-04-25T10:59:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.982+0000 7fccd17fa640 1 -- 172.21.15.19:0/2840694837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc02fbc0 con 0x7fccd410d390 2024-04-25T10:59:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.983+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd419fce0 con 0x7fccd410d390 2024-04-25T10:59:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.985+0000 7fcce139c740 1 Processor -- start 2024-04-25T10:59:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.985+0000 7fccd17fa640 1 -- 172.21.15.19:0/2840694837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fcccc02fd60 con 0x7fccd410d390 2024-04-25T10:59:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.985+0000 7fcce139c740 1 -- start start 2024-04-25T10:59:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fcce139c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b03620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fcce139c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55760173a550 0x557601b03b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fcce139c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601742730 0x557601b040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fcce139c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557601a69b00 con 0x557601763410 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fcce139c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557601a05920 con 0x55760173a550 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fcce139c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557601a65ac0 con 0x557601742730 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b03620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b03620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48110/0 (socket says 172.21.15.19:48110) 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 -- 172.21.15.19:0/998994704 learned_addr learned my addr 172.21.15.19:0/998994704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd3fff640 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55760173a550 0x557601b03b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601742730 0x557601b040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd17fa640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fccbc0afc60 0x7fccbc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 -- 172.21.15.19:0/998994704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601742730 msgr2=0x557601b040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601742730 0x557601b040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 -- 172.21.15.19:0/998994704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55760173a550 msgr2=0x557601b03b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55760173a550 0x557601b03b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 -- 172.21.15.19:0/998994704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576018437e0 con 0x557601763410 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd8a8d640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fccbc0afc60 0x7fccbc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd37fe640 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b03620 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fccc8010340 tx=0x7fccc80094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8f76ed6b57be5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.986+0000 7fccd17fa640 1 -- 172.21.15.19:0/2840694837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fcccc100700 con 0x7fccd410d390 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.987+0000 7fccba7fc640 1 -- 172.21.15.19:0/998994704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8005020 con 0x557601763410 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.987+0000 7fccba7fc640 1 -- 172.21.15.19:0/998994704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc80047a0 con 0x557601763410 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.987+0000 7fccba7fc640 1 -- 172.21.15.19:0/998994704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8013660 con 0x557601763410 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.987+0000 7fcce139c740 1 -- 172.21.15.19:0/998994704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 msgr2=0x557601b03620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.987+0000 7fccd8a8d640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fccbc0afc60 0x7fccbc0b2170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0fe480 tx=0x7fcccc02f660 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.987+0000 7fcce139c740 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b03620 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fccc8010340 tx=0x7fccc80094e0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 -- 172.21.15.19:0/998994704 shutdown_connections 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601742730 0x557601b040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55760173a550 0x557601b03b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 --2- 172.21.15.19:0/998994704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b03620 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 -- 172.21.15.19:0/998994704 >> 172.21.15.19:0/998994704 conn(0x557601757560 msgr2=0x557601affdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 -- 172.21.15.19:0/998994704 shutdown_connections 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 -- 172.21.15.19:0/998994704 wait complete. 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 -- start start 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 Processor -- start 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.988+0000 7fcce139c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55760173a550 0x557601b0dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fcce139c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b0e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fcce139c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601763410 0x557601b13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fcce139c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557601b10970 con 0x55760173a550 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fcce139c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557601a035d0 con 0x557601742730 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fcce139c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557601b10af0 con 0x557601763410 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fcce139c740 10 client.0 _ll_register_callbacks cb 0x5576019f1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55760173a550 0x557601b0dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55760173a550 0x557601b0dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48112/0 (socket says 172.21.15.19:48112) 2024-04-25T10:59:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 -- 172.21.15.19:0/1879892817 learned_addr learned my addr 172.21.15.19:0/1879892817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd3fff640 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b0e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 -- 172.21.15.19:0/1879892817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601763410 msgr2=0x557601b13e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601763410 0x557601b13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 -- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 msgr2=0x557601b0e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b0e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 -- 172.21.15.19:0/1879892817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576018a3f70 con 0x55760173a550 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd3fff640 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b0e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccd37fe640 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55760173a550 0x557601b0dce0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fccc800a010 tx=0x7fccc8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905ec441b9d49f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccb37fe640 1 -- 172.21.15.19:0/1879892817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8005000 con 0x55760173a550 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fccb37fe640 1 -- 172.21.15.19:0/1879892817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fccc80051a0 con 0x55760173a550 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.989+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc80047a0 con 0x55760173a550 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fccb37fe640 1 -- 172.21.15.19:0/1879892817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fccc8030050 con 0x55760173a550 2024-04-25T10:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 10 client.16398 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55760173b170 con 0x55760173a550 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fccb37fe640 1 -- 172.21.15.19:0/1879892817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc80047a0 con 0x55760173a550 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fccb37fe640 1 -- 172.21.15.19:0/1879892817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fccc800c030 con 0x55760173a550 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 10 client.16398 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 20 client.16398 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576017221f0 0x557601b1ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 4 client.16398 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557601a77710 con 0x5576017221f0 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.990+0000 7fccd3fff640 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576017221f0 0x557601b1ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.991+0000 7fccb37fe640 10 client.16398 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.991+0000 7fccd3fff640 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576017221f0 0x557601b1ffa0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6bada3dcfcd2d0 server_cookie=8732d50e17bd6bc in_seq=0 out_seq=0 2024-04-25T10:59:56.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.991+0000 7fccb37fe640 1 -- 172.21.15.19:0/1879892817 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557601a77710 con 0x5576017221f0 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.991+0000 7fccb37fe640 10 client.16398 handle_command_reply: tid=0 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 client.16398 shutdown 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55760173a550 msgr2=0x557601b0dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55760173a550 0x557601b0dce0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fccc800a010 tx=0x7fccc8004680 comp rx=0 tx=0).stop 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 shutdown_connections 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576017221f0 0x557601b1ffa0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557601763410 0x557601b13e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b0e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fccb37fe640 0 client.16398 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 --2- 172.21.15.19:0/1879892817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55760173a550 0x557601b0dce0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 >> 172.21.15.19:0/1879892817 conn(0x557601757560 msgr2=0x557601b033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 shutdown_connections 2024-04-25T10:59:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 1 -- 172.21.15.19:0/1879892817 wait complete. 2024-04-25T10:59:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:56.992+0000 7fcce139c740 20 client.16398 trim_cache size 0 max 16384 2024-04-25T10:59:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 Processor -- start 2024-04-25T10:59:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 -- start start 2024-04-25T10:59:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b0c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601742730 0x557601b13d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601763410 0x557601b142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41ad860 con 0x557601742730 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41b00e0 con 0x557601763410 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fcce139c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41aded0 con 0x55760173a550 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fccd3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601742730 0x557601b13d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fccd3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601742730 0x557601b13d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48116/0 (socket says 172.21.15.19:48116) 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.010+0000 7fccd3fff640 1 -- 172.21.15.19:0/894208357 learned_addr learned my addr 172.21.15.19:0/894208357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd37fe640 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601763410 0x557601b142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd37fe640 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b0c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd3fff640 1 -- 172.21.15.19:0/894208357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 msgr2=0x557601b0c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd3fff640 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b0c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd3fff640 1 -- 172.21.15.19:0/894208357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601763410 msgr2=0x557601b142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd3fff640 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601763410 0x557601b142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd3fff640 1 -- 172.21.15.19:0/894208357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557601a65a50 con 0x557601742730 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd37fe640 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601763410 0x557601b142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccd3fff640 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601742730 0x557601b13d60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005fd0 tx=0x7fccc40094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d206b4fa24a6ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccb9ffb640 1 -- 172.21.15.19:0/894208357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc401a5f0 con 0x557601742730 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccb9ffb640 1 -- 172.21.15.19:0/894208357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc401e030 con 0x557601742730 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.011+0000 7fccb9ffb640 1 -- 172.21.15.19:0/894208357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4011e00 con 0x557601742730 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 -- 172.21.15.19:0/894208357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601742730 msgr2=0x557601b13d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601742730 0x557601b13d60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005fd0 tx=0x7fccc40094a0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 -- 172.21.15.19:0/894208357 shutdown_connections 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601763410 0x557601b142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601742730 0x557601b13d60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 --2- 172.21.15.19:0/894208357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b0c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 -- 172.21.15.19:0/894208357 >> 172.21.15.19:0/894208357 conn(0x557601757560 msgr2=0x557601b0c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 -- 172.21.15.19:0/894208357 shutdown_connections 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 -- 172.21.15.19:0/894208357 wait complete. 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.012+0000 7fcce139c740 1 -- start start 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 1 Processor -- start 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b1a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b1a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b17720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41ae5a0 con 0x557601763410 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41afa10 con 0x557601742730 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41aec70 con 0x55760173a550 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fcce139c740 10 client.0 _ll_register_callbacks cb 0x7fccc8009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b17720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b17720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48124/0 (socket says 172.21.15.19:48124) 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 -- 172.21.15.19:0/358295554 learned_addr learned my addr 172.21.15.19:0/358295554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd3fff640 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b1a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b1a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 -- 172.21.15.19:0/358295554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 msgr2=0x557601b1a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b1a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 -- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 msgr2=0x557601b1a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b1a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.013+0000 7fccd37fe640 1 -- 172.21.15.19:0/358295554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557601b17e10 con 0x557601763410 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fccd37fe640 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b17720 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fccc8010d10 tx=0x7fccc8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2668e0b2c54f44b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fccb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fccb37fe640 1 -- 172.21.15.19:0/358295554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80094b0 con 0x557601763410 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fccb37fe640 1 -- 172.21.15.19:0/358295554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fccc801b040 con 0x557601763410 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc401e030 con 0x557601763410 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fccb37fe640 1 -- 172.21.15.19:0/358295554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fccc8022470 con 0x557601763410 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 10 client.16404 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557601b879a0 con 0x557601763410 2024-04-25T10:59:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fccb37fe640 1 -- 172.21.15.19:0/358295554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc401e030 con 0x557601763410 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fccb37fe640 1 -- 172.21.15.19:0/358295554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fccc800c040 con 0x557601763410 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 10 client.16404 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 20 client.16404 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576016f7630 0x557601b8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 4 client.16404 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.014+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557601a71580 con 0x5576016f7630 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.015+0000 7fccd3fff640 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576016f7630 0x557601b8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.015+0000 7fccb37fe640 10 client.16404 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.015+0000 7fccd3fff640 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576016f7630 0x557601b8a030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6403b9815102511f server_cookie=7b92e25fe25128f in_seq=0 out_seq=0 2024-04-25T10:59:57.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fccb37fe640 1 -- 172.21.15.19:0/358295554 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557601a71580 con 0x5576016f7630 2024-04-25T10:59:57.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fccb37fe640 10 client.16404 handle_command_reply: tid=0 2024-04-25T10:59:57.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 client.16404 shutdown 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 msgr2=0x557601b17720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b17720 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fccc8010d10 tx=0x7fccc8004680 comp rx=0 tx=0).stop 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 shutdown_connections 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5576016f7630 0x557601b8a030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557601763410 0x557601b17720 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fccb37fe640 0 client.16404 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557601742730 0x557601b1a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 --2- 172.21.15.19:0/358295554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55760173a550 0x557601b1a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 >> 172.21.15.19:0/358295554 conn(0x557601757560 msgr2=0x557601b033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 shutdown_connections 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 1 -- 172.21.15.19:0/358295554 wait complete. 2024-04-25T10:59:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.017+0000 7fcce139c740 20 client.16404 trim_cache size 0 max 16384 2024-04-25T10:59:57.023 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:59:57.023 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:59:57.023 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "b54e21bc-f6ba-4fcf-bf3b-2c71dced981e", 2024-04-25T10:59:57.023 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:59:57.023 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fccbc0afc60 msgr2=0x7fccbc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fccbc0afc60 0x7fccbc0b2170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0fe480 tx=0x7fcccc02f660 comp rx=0 tx=0).stop 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 msgr2=0x7fccd419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd419eec0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005cc0 tx=0x7fcccc02f380 comp rx=0 tx=0).stop 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 shutdown_connections 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fccbc0afc60 0x7fccbc0b2170 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd410d390 0x7fccd419eec0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd4108a30 0x7fccd419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 --2- 172.21.15.19:0/2840694837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd4102b00 0x7fccd419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 >> 172.21.15.19:0/2840694837 conn(0x7fccd40fe260 msgr2=0x7fccd4074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 shutdown_connections 2024-04-25T10:59:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.018+0000 7fccda4f0640 1 -- 172.21.15.19:0/2840694837 wait complete. 2024-04-25T10:59:57.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:59:57.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 -- 172.21.15.19:0/1955502660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 msgr2=0x7fa7601036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 --2- 172.21.15.19:0/1955502660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa7601036b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa750009900 tx=0x7fa75002a410 comp rx=0 tx=0).stop 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 -- 172.21.15.19:0/1955502660 shutdown_connections 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 --2- 172.21.15.19:0/1955502660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa760104260 0x7fa760110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 --2- 172.21.15.19:0/1955502660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa760109fb0 0x7fa760103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 --2- 172.21.15.19:0/1955502660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa7601036b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 -- 172.21.15.19:0/1955502660 >> 172.21.15.19:0/1955502660 conn(0x7fa7600ff1d0 msgr2=0x7fa760101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 -- 172.21.15.19:0/1955502660 shutdown_connections 2024-04-25T10:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.184+0000 7fa764f84640 1 -- 172.21.15.19:0/1955502660 wait complete. 2024-04-25T10:59:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 Processor -- start 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 -- start start 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa760104260 0x7fa76019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa76019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa760109fb0 0x7fa76019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa760113d00 con 0x7fa760109b10 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa760113b80 con 0x7fa760104260 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa764f84640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa760113e80 con 0x7fa760109fb0 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa75dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa76019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa75dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa76019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48140/0 (socket says 172.21.15.19:48140) 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.185+0000 7fa75dd9b640 1 -- 172.21.15.19:0/1946675234 learned_addr learned my addr 172.21.15.19:0/1946675234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa760109fb0 0x7fa76019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75e59c640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa760104260 0x7fa76019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75dd9b640 1 -- 172.21.15.19:0/1946675234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa760109fb0 msgr2=0x7fa76019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa760109fb0 0x7fa76019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75dd9b640 1 -- 172.21.15.19:0/1946675234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa760104260 msgr2=0x7fa76019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa760104260 0x7fa76019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75dd9b640 1 -- 172.21.15.19:0/1946675234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76019f540 con 0x7fa760109b10 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa760109fb0 0x7fa76019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa76019ac40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa75400e9f0 tx=0x7fa75400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa74f7fe640 1 -- 172.21.15.19:0/1946675234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75400ccc0 con 0x7fa760109b10 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa74f7fe640 1 -- 172.21.15.19:0/1946675234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7540035f0 con 0x7fa760109b10 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76019f830 con 0x7fa760109b10 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa74f7fe640 1 -- 172.21.15.19:0/1946675234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75400ce60 con 0x7fa760109b10 2024-04-25T10:59:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.186+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601a71b0 con 0x7fa760109b10 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 Processor -- start 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 -- start start 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa74f7fe640 1 -- 172.21.15.19:0/1946675234 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa754003aa0 con 0x7fa760109b10 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c0a615d80 con 0x558c0a2e64d0 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c0a5cf8b0 con 0x558c0a2ee6b0 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa76be3b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558c0a611d40 con 0x558c0a30f390 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa75e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa75e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48154/0 (socket says 172.21.15.19:48154) 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa75e59c640 1 -- 172.21.15.19:0/1807252191 learned_addr learned my addr 172.21.15.19:0/1807252191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.189+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75e59c640 1 -- 172.21.15.19:0/1807252191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 msgr2=0x558c0a6af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75e59c640 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75e59c640 1 -- 172.21.15.19:0/1807252191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 msgr2=0x558c0a6c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75e59c640 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75e59c640 1 -- 172.21.15.19:0/1807252191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0a5a0a40 con 0x558c0a2e64d0 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa74f7fe640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa7480afc60 0x7fa7480b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75e59c640 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6afe70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa750009900 tx=0x7fa750032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695fc170f5ac71ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa7480afc60 0x7fa7480b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa74cff9640 1 -- 172.21.15.19:0/1807252191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75003f070 con 0x558c0a2e64d0 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa74f7fe640 1 -- 172.21.15.19:0/1946675234 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa7540df3f0 con 0x7fa760109b10 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa74cff9640 1 -- 172.21.15.19:0/1807252191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa750004030 con 0x558c0a2e64d0 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.190+0000 7fa74cff9640 1 -- 172.21.15.19:0/1807252191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750035070 con 0x558c0a2e64d0 2024-04-25T10:59:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 -- 172.21.15.19:0/1807252191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 msgr2=0x558c0a6afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa7480afc60 0x7fa7480b2170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa7580095f0 tx=0x7fa758009580 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6afe70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa750009900 tx=0x7fa750032ba0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 -- 172.21.15.19:0/1807252191 shutdown_connections 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6afe70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 --2- 172.21.15.19:0/1807252191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 -- 172.21.15.19:0/1807252191 >> 172.21.15.19:0/1807252191 conn(0x558c0a3034e0 msgr2=0x558c0a6ac210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 -- 172.21.15.19:0/1807252191 shutdown_connections 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.191+0000 7fa76be3b740 1 -- 172.21.15.19:0/1807252191 wait complete. 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 -- start start 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 Processor -- start 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6be9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c0a6c1420 con 0x558c0a2ee6b0 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c0a59cd10 con 0x558c0a2e64d0 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558c0a6c15a0 con 0x558c0a30f390 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa76be3b740 10 client.0 _ll_register_callbacks cb 0x558c0a5c9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48168/0 (socket says 172.21.15.19:48168) 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 -- 172.21.15.19:0/2956632580 learned_addr learned my addr 172.21.15.19:0/2956632580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75e59c640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6be9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 -- 172.21.15.19:0/2956632580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 msgr2=0x558c0a6c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 -- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 msgr2=0x558c0a6be9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6be9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75dd9b640 1 -- 172.21.15.19:0/2956632580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0a3f17d0 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.192+0000 7fa75e59c640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6be9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bef20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa7540a7550 tx=0x7fa7540a3a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b0804e16b06c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa745ffb640 1 -- 172.21.15.19:0/2956632580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7540a6f10 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa745ffb640 1 -- 172.21.15.19:0/2956632580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa7540aff90 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa750004030 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa745ffb640 1 -- 172.21.15.19:0/2956632580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa7540e2e60 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 10 client.16416 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558c0a3273f0 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa745ffb640 1 -- 172.21.15.19:0/2956632580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa750004030 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa745ffb640 1 -- 172.21.15.19:0/2956632580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa7540a2f30 con 0x558c0a2ee6b0 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 10 client.16416 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 20 client.16416 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2cdf60 0x558c0a6d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 4 client.16416 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.193+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c0a623990 con 0x558c0a2cdf60 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.194+0000 7fa75e59c640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2cdf60 0x558c0a6d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.194+0000 7fa745ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.194+0000 7fa75e59c640 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2cdf60 0x558c0a6d0950 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91541b2c0c39bf55 server_cookie=5f82d5848f163004 in_seq=0 out_seq=0 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa745ffb640 1 -- 172.21.15.19:0/2956632580 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c0a623990 con 0x558c0a2cdf60 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa745ffb640 10 client.16416 handle_command_reply: tid=0 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 client.16416 shutdown 2024-04-25T10:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2ee6b0 msgr2=0x558c0a6bef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bef20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa7540a7550 tx=0x7fa7540a3a80 comp rx=0 tx=0).stop 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 shutdown_connections 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2cdf60 0x558c0a6d0950 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bef20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa745ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 --2- 172.21.15.19:0/2956632580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6be9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 >> 172.21.15.19:0/2956632580 conn(0x558c0a3034e0 msgr2=0x558c0a6b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 shutdown_connections 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 1 -- 172.21.15.19:0/2956632580 wait complete. 2024-04-25T10:59:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.195+0000 7fa76be3b740 20 client.16416 trim_cache size 0 max 16384 2024-04-25T10:59:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 Processor -- start 2024-04-25T10:59:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 -- start start 2024-04-25T10:59:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a30f390 0x558c0a6d60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601ab6d0 con 0x558c0a30f390 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601adf50 con 0x558c0a2e64d0 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa76be3b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601abd40 con 0x558c0a2ee6b0 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a30f390 0x558c0a6d60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a30f390 0x558c0a6d60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48174/0 (socket says 172.21.15.19:48174) 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 -- 172.21.15.19:0/3346521391 learned_addr learned my addr 172.21.15.19:0/3346521391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75e59c640 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 -- 172.21.15.19:0/3346521391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a2ee6b0 msgr2=0x558c0a6bd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 -- 172.21.15.19:0/3346521391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 msgr2=0x558c0a6bf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6bf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.214+0000 7fa75ed9d640 1 -- 172.21.15.19:0/3346521391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0a2e70f0 con 0x558c0a30f390 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a30f390 0x558c0a6d60e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa75801c610 tx=0x7fa758015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2998e90d4576b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa747fff640 1 -- 172.21.15.19:0/3346521391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75802c070 con 0x558c0a30f390 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa747fff640 1 -- 172.21.15.19:0/3346521391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa758010a60 con 0x558c0a30f390 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa747fff640 1 -- 172.21.15.19:0/3346521391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758027a60 con 0x558c0a30f390 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa76be3b740 1 -- 172.21.15.19:0/3346521391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a30f390 msgr2=0x558c0a6d60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa76be3b740 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a30f390 0x558c0a6d60e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa75801c610 tx=0x7fa758015000 comp rx=0 tx=0).stop 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa76be3b740 1 -- 172.21.15.19:0/3346521391 shutdown_connections 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa76be3b740 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a30f390 0x558c0a6d60e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa76be3b740 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a2ee6b0 0x558c0a6bd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa76be3b740 1 --2- 172.21.15.19:0/3346521391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2e64d0 0x558c0a6bf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.215+0000 7fa76be3b740 1 -- 172.21.15.19:0/3346521391 >> 172.21.15.19:0/3346521391 conn(0x558c0a3034e0 msgr2=0x558c0a6bd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 -- 172.21.15.19:0/3346521391 shutdown_connections 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 -- 172.21.15.19:0/3346521391 wait complete. 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 -- start start 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 Processor -- start 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601ac410 con 0x558c0a2e64d0 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601ad880 con 0x558c0a2ee6b0 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601acae0 con 0x558c0a30f390 2024-04-25T10:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa76be3b740 10 client.0 _ll_register_callbacks cb 0x7fa75400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa75e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa75e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48182/0 (socket says 172.21.15.19:48182) 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa75e59c640 1 -- 172.21.15.19:0/275392729 learned_addr learned my addr 172.21.15.19:0/275392729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.216+0000 7fa75ed9d640 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa75e59c640 1 -- 172.21.15.19:0/275392729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 msgr2=0x558c0a6c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa75e59c640 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa75e59c640 1 -- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 msgr2=0x558c0a6cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa75e59c640 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa75e59c640 1 -- 172.21.15.19:0/275392729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0a5d64b0 con 0x558c0a2e64d0 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa75e59c640 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6cad10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa750005f70 tx=0x7fa7500316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcff3849bbadc53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:59:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa745ffb640 1 -- 172.21.15.19:0/275392729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750052020 con 0x558c0a2e64d0 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa745ffb640 1 -- 172.21.15.19:0/275392729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa750031d10 con 0x558c0a2e64d0 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa758010a60 con 0x558c0a2e64d0 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa745ffb640 1 -- 172.21.15.19:0/275392729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa75005d050 con 0x558c0a2e64d0 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa76be3b740 10 client.16422 fetch_fsmap learned FSMap version 20 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558c0a738400 con 0x558c0a2e64d0 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.217+0000 7fa745ffb640 1 -- 172.21.15.19:0/275392729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa758010a60 con 0x558c0a2e64d0 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa745ffb640 1 -- 172.21.15.19:0/275392729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa75004e5b0 con 0x558c0a2e64d0 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa76be3b740 10 client.16422 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa76be3b740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa76be3b740 20 client.16422 populate_metadata read hostname 'smithi019' 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa76be3b740 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2a33a0 0x558c0a73a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa76be3b740 4 client.16422 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:59:57.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c0a61d800 con 0x558c0a2a33a0 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2a33a0 0x558c0a73a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa745ffb640 10 client.16422 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa75dd9b640 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2a33a0 0x558c0a73a760 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73448eda66ecd924 server_cookie=2909e19a6b4df38e in_seq=0 out_seq=0 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.218+0000 7fa745ffb640 1 -- 172.21.15.19:0/275392729 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558c0a61d800 con 0x558c0a2a33a0 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa745ffb640 10 client.16422 handle_command_reply: tid=0 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 client.16422 shutdown 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 msgr2=0x558c0a6cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6cad10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa750005f70 tx=0x7fa7500316d0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 shutdown_connections 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558c0a2a33a0 0x558c0a73a760 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c0a30f390 0x558c0a6c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c0a2ee6b0 0x558c0a6cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa745ffb640 0 client.16422 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 --2- 172.21.15.19:0/275392729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0a2e64d0 0x558c0a6cad10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 >> 172.21.15.19:0/275392729 conn(0x558c0a3034e0 msgr2=0x558c0a6bfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 shutdown_connections 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 1 -- 172.21.15.19:0/275392729 wait complete. 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa76be3b740 20 client.16422 trim_cache size 0 max 16384 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:59:57.227 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: "b54e21bc-f6ba-4fcf-bf3b-2c71dced981e": { 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "b54e21bc-f6ba-4fcf-bf3b-2c71dced981e", 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:59:57.228 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:59:57.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa7480afc60 msgr2=0x7fa7480b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa7480afc60 0x7fa7480b2170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa7580095f0 tx=0x7fa758009580 comp rx=0 tx=0).stop 2024-04-25T10:59:57.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 msgr2=0x7fa76019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa76019ac40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa75400e9f0 tx=0x7fa75400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 shutdown_connections 2024-04-25T10:59:57.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa7480afc60 0x7fa7480b2170 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa760109fb0 0x7fa76019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760109b10 0x7fa76019ac40 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 --2- 172.21.15.19:0/1946675234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa760104260 0x7fa76019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.219+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 >> 172.21.15.19:0/1946675234 conn(0x7fa7600ff1d0 msgr2=0x7fa7600ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.220+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 shutdown_connections 2024-04-25T10:59:57.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:59:57.220+0000 7fa764f84640 1 -- 172.21.15.19:0/1946675234 wait complete. 2024-04-25T10:59:57.231 INFO:tasks.cephfs.filesystem:scrub status for tag:b54e21bc-f6ba-4fcf-bf3b-2c71dced981e - {'path': '/', 'tag': 'b54e21bc-f6ba-4fcf-bf3b-2c71dced981e', 'options': 'recursive,force'} 2024-04-25T10:59:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:57 smithi019 ceph-mon[29209]: pgmap v166: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:57 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T10:59:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:57 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:57 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T10:59:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:57 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:57 smithi169 ceph-mon[33049]: pgmap v166: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:57 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T10:59:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:57 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:57 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T10:59:57.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:57 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:57.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:57 smithi112 ceph-mon[32799]: pgmap v166: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T10:59:57.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:57 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T10:59:57.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:57 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:57.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:57 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T10:59:57.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:57 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T10:59:59.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 10:59:59 smithi169 ceph-mon[33049]: pgmap v167: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:59.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 10:59:59 smithi112 ceph-mon[32799]: pgmap v167: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T10:59:59.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 10:59:59 smithi019 ceph-mon[29209]: pgmap v167: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:00.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:00 smithi169 ceph-mon[33049]: overall HEALTH_OK 2024-04-25T11:00:00.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:00 smithi112 ceph-mon[32799]: overall HEALTH_OK 2024-04-25T11:00:00.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:00 smithi019 ceph-mon[29209]: overall HEALTH_OK 2024-04-25T11:00:01.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:01 smithi169 ceph-mon[33049]: pgmap v168: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:01.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:01 smithi112 ceph-mon[32799]: pgmap v168: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:01.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:01 smithi019 ceph-mon[29209]: pgmap v168: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:02.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:02 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:00:02.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:02 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:00:02.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:02 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:00:03.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:03 smithi112 ceph-mon[32799]: pgmap v169: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:03.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:03 smithi169 ceph-mon[33049]: pgmap v169: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:03.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:03 smithi019 ceph-mon[29209]: pgmap v169: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:05.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:05 smithi169 ceph-mon[33049]: pgmap v170: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:05.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:05 smithi112 ceph-mon[32799]: pgmap v170: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:05.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:05 smithi019 ceph-mon[29209]: pgmap v170: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:07 smithi169 ceph-mon[33049]: pgmap v171: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:07.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:07 smithi112 ceph-mon[32799]: pgmap v171: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:07.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:07 smithi019 ceph-mon[29209]: pgmap v171: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:09.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:09 smithi169 ceph-mon[33049]: pgmap v172: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:09.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:09 smithi112 ceph-mon[32799]: pgmap v172: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:09.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:09 smithi019 ceph-mon[29209]: pgmap v172: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:11.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:11 smithi169 ceph-mon[33049]: pgmap v173: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:11.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:11 smithi112 ceph-mon[32799]: pgmap v173: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:11.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:11 smithi019 ceph-mon[29209]: pgmap v173: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:00:13.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:13 smithi169 ceph-mon[33049]: pgmap v174: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:13.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:13 smithi112 ceph-mon[32799]: pgmap v174: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:13.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:13 smithi019 ceph-mon[29209]: pgmap v174: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-04-25T11:00:15.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:15 smithi169 ceph-mon[33049]: pgmap v175: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:00:15.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:15 smithi112 ceph-mon[32799]: pgmap v175: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:00:15.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:15 smithi019 ceph-mon[29209]: pgmap v175: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:00:17.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:17 smithi169 ceph-mon[33049]: pgmap v176: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:00:17.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:17 smithi112 ceph-mon[32799]: pgmap v176: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:00:17.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:17 smithi019 ceph-mon[29209]: pgmap v176: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:00:19.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:19 smithi019 ceph-mon[29209]: pgmap v177: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:00:19.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:19 smithi169 ceph-mon[33049]: pgmap v177: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:00:19.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:19 smithi112 ceph-mon[32799]: pgmap v177: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:00:21.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:21 smithi169 ceph-mon[33049]: pgmap v178: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:00:21.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:21 smithi112 ceph-mon[32799]: pgmap v178: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:00:21.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:21 smithi019 ceph-mon[29209]: pgmap v178: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:00:23.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:23 smithi169 ceph-mon[33049]: pgmap v179: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:00:23.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:23 smithi112 ceph-mon[32799]: pgmap v179: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:00:23.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:23 smithi019 ceph-mon[29209]: pgmap v179: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:00:25.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:25 smithi169 ceph-mon[33049]: pgmap v180: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:25.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:25 smithi112 ceph-mon[32799]: pgmap v180: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:25.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:25 smithi019 ceph-mon[29209]: pgmap v180: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:27.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:00:27.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 -- 172.21.15.19:0/4200658574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc109fb0 msgr2=0x7fd8bc103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/4200658574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc109fb0 0x7fd8bc103bf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009900 tx=0x7fd8a402a410 comp rx=0 tx=0).stop 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 -- 172.21.15.19:0/4200658574 shutdown_connections 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/4200658574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8bc104260 0x7fd8bc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/4200658574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc109fb0 0x7fd8bc103bf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/4200658574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8bc109b10 0x7fd8bc1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 -- 172.21.15.19:0/4200658574 >> 172.21.15.19:0/4200658574 conn(0x7fd8bc0ff1d0 msgr2=0x7fd8bc101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 -- 172.21.15.19:0/4200658574 shutdown_connections 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.363+0000 7fd8c21b2640 1 -- 172.21.15.19:0/4200658574 wait complete. 2024-04-25T11:00:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 Processor -- start 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 -- start start 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc104260 0x7fd8bc075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8bc109b10 0x7fd8bc0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8bc109fb0 0x7fd8bc076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc113d00 con 0x7fd8bc104260 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc113b80 con 0x7fd8bc109fb0 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8c21b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc113e80 con 0x7fd8bc109b10 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc104260 0x7fd8bc075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc104260 0x7fd8bc075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51826/0 (socket says 172.21.15.19:51826) 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/2038738454 learned_addr learned my addr 172.21.15.19:0/2038738454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8bc109fb0 0x7fd8bc076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.364+0000 7fd8baffd640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8bc109b10 0x7fd8bc0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/2038738454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8bc109b10 msgr2=0x7fd8bc0764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8bc109b10 0x7fd8bc0764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/2038738454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8bc109fb0 msgr2=0x7fd8bc076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8bc109fb0 0x7fd8bc076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/2038738454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8bc1a8ac0 con 0x7fd8bc104260 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8bc109fb0 0x7fd8bc076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc104260 0x7fd8bc075fb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00ebc0 tx=0x7fd8ac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8b8ff9640 1 -- 172.21.15.19:0/2038738454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac004d20 con 0x7fd8bc104260 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8b8ff9640 1 -- 172.21.15.19:0/2038738454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8ac0054d0 con 0x7fd8bc104260 2024-04-25T11:00:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8bc1a8db0 con 0x7fd8bc104260 2024-04-25T11:00:27.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8b8ff9640 1 -- 172.21.15.19:0/2038738454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac010720 con 0x7fd8bc104260 2024-04-25T11:00:27.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.365+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8bc1a91c0 con 0x7fd8bc104260 2024-04-25T11:00:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 Processor -- start 2024-04-25T11:00:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8b8ff9640 1 -- 172.21.15.19:0/2038738454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd8ac005020 con 0x7fd8bc104260 2024-04-25T11:00:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 -- start start 2024-04-25T11:00:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e7ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e8b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562538de4d80 con 0x562538ab54d0 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562538d9e8b0 con 0x562538abd6b0 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8c9063740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562538de0d40 con 0x562538ade390 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e7efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51836/0 (socket says 172.21.15.19:51836) 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/1610069416 learned_addr learned my addr 172.21.15.19:0/1610069416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8baffd640 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e8b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e7ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/1610069416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 msgr2=0x562538e7ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e7ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/1610069416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 msgr2=0x562538e8b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e8b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8b8ff9640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd8a80afc60 0x7fd8a80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.368+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/1610069416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562538d6fa40 con 0x562538ab54d0 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.369+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd8a80afc60 0x7fd8a80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.369+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e7efa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0dd0a0 tx=0x7fd8ac0e3c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85aa26c4b539538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.369+0000 7fd8b8ff9640 1 -- 172.21.15.19:0/2038738454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd8ac0df9b0 con 0x7fd8bc104260 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.369+0000 7fd8b1ffb640 1 -- 172.21.15.19:0/1610069416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0dfe60 con 0x562538ab54d0 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.369+0000 7fd8b1ffb640 1 -- 172.21.15.19:0/1610069416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8ac005310 con 0x562538ab54d0 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.369+0000 7fd8b1ffb640 1 -- 172.21.15.19:0/1610069416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0a3a40 con 0x562538ab54d0 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 -- 172.21.15.19:0/1610069416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 msgr2=0x562538e7efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd8a80afc60 0x7fd8a80b2170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x562538e7bbd0 tx=0x7fd8b4005f30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e7efa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0dd0a0 tx=0x7fd8ac0e3c80 comp rx=0 tx=0).stop 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 -- 172.21.15.19:0/1610069416 shutdown_connections 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e8b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e7efa0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 --2- 172.21.15.19:0/1610069416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e7ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 -- 172.21.15.19:0/1610069416 >> 172.21.15.19:0/1610069416 conn(0x562538ad24e0 msgr2=0x562538e7b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 -- 172.21.15.19:0/1610069416 shutdown_connections 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 -- 172.21.15.19:0/1610069416 wait complete. 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.370+0000 7fd8c9063740 1 -- start start 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 1 Processor -- start 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e89650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e89b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e8f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562538e8bfe0 con 0x562538ab54d0 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562538d6bd10 con 0x562538abd6b0 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562538e8c160 con 0x562538ade390 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8c9063740 10 client.0 _ll_register_callbacks cb 0x562538d98210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e89650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e89650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51850/0 (socket says 172.21.15.19:51850) 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/3170700080 learned_addr learned my addr 172.21.15.19:0/3170700080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e8f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8baffd640 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e89b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/3170700080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 msgr2=0x562538e8f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e8f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 msgr2=0x562538e89b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e89b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.371+0000 7fd8bb7fe640 1 -- 172.21.15.19:0/3170700080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562538bc07d0 con 0x562538ab54d0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e89650 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0ca040 tx=0x7fd8ac0dfcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba7b081adcc9e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd89affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd89affd640 1 -- 172.21.15.19:0/3170700080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0dfe60 con 0x562538ab54d0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd89affd640 1 -- 172.21.15.19:0/3170700080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd8ac0a3e10 con 0x562538ab54d0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562538df2990 con 0x562538ab54d0 2024-04-25T11:00:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd89affd640 1 -- 172.21.15.19:0/3170700080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd8ac0a72f0 con 0x562538ab54d0 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 10 client.16434 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562538af63f0 con 0x562538ab54d0 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd89affd640 1 -- 172.21.15.19:0/3170700080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562538df2990 con 0x562538ab54d0 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd89affd640 1 -- 172.21.15.19:0/3170700080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd8ac0a7de0 con 0x562538ab54d0 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 10 client.16434 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 20 client.16434 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538a9cf60 0x562538e9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 4 client.16434 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.372+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562538dec800 con 0x562538a9cf60 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.373+0000 7fd8baffd640 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538a9cf60 0x562538e9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.373+0000 7fd8baffd640 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538a9cf60 0x562538e9b5c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20303ed7fe1b151b server_cookie=39eb69ee35426927 in_seq=0 out_seq=0 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.373+0000 7fd89affd640 10 client.16434 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd89affd640 1 -- 172.21.15.19:0/3170700080 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562538dec800 con 0x562538a9cf60 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd89affd640 10 client.16434 handle_command_reply: tid=0 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 client.16434 shutdown 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 msgr2=0x562538e89650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e89650 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0ca040 tx=0x7fd8ac0dfcb0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 shutdown_connections 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538a9cf60 0x562538e9b5c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538ade390 0x562538e8f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538abd6b0 0x562538e89b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd89affd640 0 client.16434 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 --2- 172.21.15.19:0/3170700080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538ab54d0 0x562538e89650 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 >> 172.21.15.19:0/3170700080 conn(0x562538ad24e0 msgr2=0x562538e80780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 shutdown_connections 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.374+0000 7fd8c9063740 1 -- 172.21.15.19:0/3170700080 wait complete. 2024-04-25T11:00:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.375+0000 7fd8c9063740 20 client.16434 trim_cache size 0 max 16384 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 Processor -- start 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 -- start start 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e88380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e8b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e8b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1afa30 con 0x562538abd6b0 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b22b0 con 0x562538ab54d0 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8c9063740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b00a0 con 0x562538a9cf60 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e8b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e8b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51852/0 (socket says 172.21.15.19:51852) 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 -- 172.21.15.19:0/950331265 learned_addr learned my addr 172.21.15.19:0/950331265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8baffd640 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e8b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e88380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 -- 172.21.15.19:0/950331265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 msgr2=0x562538e88380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e88380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 -- 172.21.15.19:0/950331265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 msgr2=0x562538e8b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e8b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bbfff640 1 -- 172.21.15.19:0/950331265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562538ab60f0 con 0x562538abd6b0 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.393+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e88380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e8b660 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4012640 tx=0x7fd8b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e8699a993a45e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8b17fa640 1 -- 172.21.15.19:0/950331265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4005220 con 0x562538abd6b0 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8b17fa640 1 -- 172.21.15.19:0/950331265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b4004c50 con 0x562538abd6b0 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8b17fa640 1 -- 172.21.15.19:0/950331265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4020af0 con 0x562538abd6b0 2024-04-25T11:00:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8c9063740 1 -- 172.21.15.19:0/950331265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 msgr2=0x562538e8b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8c9063740 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e8b660 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4012640 tx=0x7fd8b4014000 comp rx=0 tx=0).stop 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8c9063740 1 -- 172.21.15.19:0/950331265 shutdown_connections 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8c9063740 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e8b660 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8c9063740 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e8b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.394+0000 7fd8c9063740 1 --2- 172.21.15.19:0/950331265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e88380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 -- 172.21.15.19:0/950331265 >> 172.21.15.19:0/950331265 conn(0x562538ad24e0 msgr2=0x562538e8aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 -- 172.21.15.19:0/950331265 shutdown_connections 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 -- 172.21.15.19:0/950331265 wait complete. 2024-04-25T11:00:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 -- start start 2024-04-25T11:00:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 Processor -- start 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e91f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e92450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e92990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b0770 con 0x562538abd6b0 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b1be0 con 0x562538ab54d0 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.395+0000 7fd8c9063740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b0e40 con 0x562538a9cf60 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8c9063740 10 client.0 _ll_register_callbacks cb 0x7fd8ac0e3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e92990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e92990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51858/0 (socket says 172.21.15.19:51858) 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 -- 172.21.15.19:0/223046236 learned_addr learned my addr 172.21.15.19:0/223046236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8baffd640 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e92450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e91f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 -- 172.21.15.19:0/223046236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 msgr2=0x562538e91f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e91f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 -- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 msgr2=0x562538e92450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e92450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 -- 172.21.15.19:0/223046236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562538da54b0 con 0x562538abd6b0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8bbfff640 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e92990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4020670 tx=0x7fd8b400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b48731f35db25b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd89affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd89affd640 1 -- 172.21.15.19:0/223046236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4035070 con 0x562538abd6b0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd89affd640 1 -- 172.21.15.19:0/223046236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd8b4004950 con 0x562538abd6b0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.396+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b4004c50 con 0x562538abd6b0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd89affd640 1 -- 172.21.15.19:0/223046236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd8b4004e00 con 0x562538abd6b0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 10 client.16440 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562538f02800 con 0x562538abd6b0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd89affd640 1 -- 172.21.15.19:0/223046236 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b4004c50 con 0x562538abd6b0 2024-04-25T11:00:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd89affd640 1 -- 172.21.15.19:0/223046236 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd8b4027070 con 0x562538abd6b0 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 10 client.16440 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 20 client.16440 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538ade390 0x562538f04ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 4 client.16440 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562538d674a0 con 0x562538ade390 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.397+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538ade390 0x562538f04ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd89affd640 10 client.16440 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8bb7fe640 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538ade390 0x562538f04ed0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7cdc1f54c04b6 server_cookie=f9b1e237b431de73 in_seq=0 out_seq=0 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd89affd640 1 -- 172.21.15.19:0/223046236 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562538d674a0 con 0x562538ade390 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd89affd640 10 client.16440 handle_command_reply: tid=0 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 client.16440 shutdown 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 msgr2=0x562538e92990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e92990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4020670 tx=0x7fd8b400bf00 comp rx=0 tx=0).stop 2024-04-25T11:00:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 shutdown_connections 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x562538ade390 0x562538f04ed0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562538abd6b0 0x562538e92990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562538ab54d0 0x562538e92450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd89affd640 0 client.16440 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 --2- 172.21.15.19:0/223046236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562538a9cf60 0x562538e91f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 >> 172.21.15.19:0/223046236 conn(0x562538ad24e0 msgr2=0x562538e8b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 shutdown_connections 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 1 -- 172.21.15.19:0/223046236 wait complete. 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.398+0000 7fd8c9063740 20 client.16440 trim_cache size 0 max 16384 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:00:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd8a80afc60 msgr2=0x7fd8a80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd8a80afc60 0x7fd8a80b2170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x562538e7bbd0 tx=0x7fd8b4005f30 comp rx=0 tx=0).stop 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc104260 msgr2=0x7fd8bc075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc104260 0x7fd8bc075fb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00ebc0 tx=0x7fd8ac00c6a0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 shutdown_connections 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd8a80afc60 0x7fd8a80b2170 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8bc109fb0 0x7fd8bc076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8bc109b10 0x7fd8bc0764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 --2- 172.21.15.19:0/2038738454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8bc104260 0x7fd8bc075fb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 >> 172.21.15.19:0/2038738454 conn(0x7fd8bc0ff1d0 msgr2=0x7fd8bc10f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 shutdown_connections 2024-04-25T11:00:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.399+0000 7fd8c21b2640 1 -- 172.21.15.19:0/2038738454 wait complete. 2024-04-25T11:00:27.410 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:00:27.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 -- 172.21.15.19:0/2386935983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 msgr2=0x7f9480105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 --2- 172.21.15.19:0/2386935983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f9480105000 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f947c009900 tx=0x7f947c02a410 comp rx=0 tx=0).stop 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 -- 172.21.15.19:0/2386935983 shutdown_connections 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 --2- 172.21.15.19:0/2386935983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948010e040 0x7f9480110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 --2- 172.21.15.19:0/2386935983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f948010b650 0x7f948010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 --2- 172.21.15.19:0/2386935983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f9480105000 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 -- 172.21.15.19:0/2386935983 >> 172.21.15.19:0/2386935983 conn(0x7f9480078110 msgr2=0x7f9480078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 -- 172.21.15.19:0/2386935983 shutdown_connections 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.522+0000 7f948729f640 1 -- 172.21.15.19:0/2386935983 wait complete. 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 Processor -- start 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 -- start start 2024-04-25T11:00:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f94801024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948010b650 0x7f94801029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f948010e040 0x7f9480102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9480113ca0 con 0x7f9480104bf0 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9480113b20 con 0x7f948010e040 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948729f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9480113e20 con 0x7f948010b650 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948503b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f94801024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948503b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f94801024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51866/0 (socket says 172.21.15.19:51866) 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.523+0000 7f948503b640 1 -- 172.21.15.19:0/390787934 learned_addr learned my addr 172.21.15.19:0/390787934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948583c640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f948010e040 0x7f9480102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948503b640 1 -- 172.21.15.19:0/390787934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948010b650 msgr2=0x7f94801029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948483a640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948010b650 0x7f94801029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948503b640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948010b650 0x7f94801029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948503b640 1 -- 172.21.15.19:0/390787934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f948010e040 msgr2=0x7f9480102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948503b640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f948010e040 0x7f9480102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948503b640 1 -- 172.21.15.19:0/390787934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801a89e0 con 0x7f9480104bf0 2024-04-25T11:00:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948583c640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f948010e040 0x7f9480102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948503b640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f94801024b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f947c005940 tx=0x7f947c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f94767fc640 1 -- 172.21.15.19:0/390787934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c03f030 con 0x7f9480104bf0 2024-04-25T11:00:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801a8c70 con 0x7f9480104bf0 2024-04-25T11:00:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f94767fc640 1 -- 172.21.15.19:0/390787934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947c003cb0 con 0x7f9480104bf0 2024-04-25T11:00:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f94767fc640 1 -- 172.21.15.19:0/390787934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c035070 con 0x7f9480104bf0 2024-04-25T11:00:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.524+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a9160 con 0x7f9480104bf0 2024-04-25T11:00:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.525+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948006a770 con 0x7f9480104bf0 2024-04-25T11:00:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.527+0000 7f94767fc640 1 -- 172.21.15.19:0/390787934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f947c042030 con 0x7f9480104bf0 2024-04-25T11:00:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.527+0000 7f94767fc640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f946c0afc60 0x7f946c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.527+0000 7f948483a640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f946c0afc60 0x7f946c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.527+0000 7f94767fc640 1 -- 172.21.15.19:0/390787934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f947c0ffc00 con 0x7f9480104bf0 2024-04-25T11:00:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.528+0000 7f948483a640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f946c0afc60 0x7f946c0b2170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9470005fd0 tx=0x7f9470005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.531+0000 7f94767fc640 1 -- 172.21.15.19:0/390787934 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f947c0c84d0 con 0x7f9480104bf0 2024-04-25T11:00:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:27 smithi019 ceph-mon[29209]: pgmap v181: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:27.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:27 smithi169 ceph-mon[33049]: pgmap v181: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.758+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f948006afa0 con 0x7f9480104bf0 2024-04-25T11:00:27.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.759+0000 7f94767fc640 1 -- 172.21.15.19:0/390787934 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f947c0c7dd0 con 0x7f9480104bf0 2024-04-25T11:00:27.759 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:00:27.759 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:00:27.760 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:00:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f946c0afc60 msgr2=0x7f946c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f946c0afc60 0x7f946c0b2170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9470005fd0 tx=0x7f9470005850 comp rx=0 tx=0).stop 2024-04-25T11:00:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 msgr2=0x7f94801024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f94801024b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f947c005940 tx=0x7f947c0042f0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 shutdown_connections 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f946c0afc60 0x7f946c0b2170 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f948010e040 0x7f9480102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948010b650 0x7f94801029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 --2- 172.21.15.19:0/390787934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480104bf0 0x7f94801024b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 >> 172.21.15.19:0/390787934 conn(0x7f9480078110 msgr2=0x7f9480101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 shutdown_connections 2024-04-25T11:00:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.761+0000 7f948729f640 1 -- 172.21.15.19:0/390787934 wait complete. 2024-04-25T11:00:27.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:00:27.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:27 smithi112 ceph-mon[32799]: pgmap v181: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 -- 172.21.15.19:0/2090811956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 msgr2=0x7f3738103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2090811956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f3738103c00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f372c009900 tx=0x7f372c02a410 comp rx=0 tx=0).stop 2024-04-25T11:00:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 -- 172.21.15.19:0/2090811956 shutdown_connections 2024-04-25T11:00:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2090811956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3738104270 0x7f3738110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2090811956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f3738103c00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2090811956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3738109b20 0x7f37381036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 -- 172.21.15.19:0/2090811956 >> 172.21.15.19:0/2090811956 conn(0x7f37380ff1e0 msgr2=0x7f3738101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 -- 172.21.15.19:0/2090811956 shutdown_connections 2024-04-25T11:00:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.888+0000 7f373cfa8640 1 -- 172.21.15.19:0/2090811956 wait complete. 2024-04-25T11:00:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 Processor -- start 2024-04-25T11:00:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 -- start start 2024-04-25T11:00:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3738104270 0x7f373819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3738109b20 0x7f373819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f373819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3738113d10 con 0x7f3738109fc0 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3738113b90 con 0x7f3738109b20 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373cfa8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3738113e90 con 0x7f3738104270 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f373819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f373819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51896/0 (socket says 172.21.15.19:51896) 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 -- 172.21.15.19:0/2997439787 learned_addr learned my addr 172.21.15.19:0/2997439787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3735d9b640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3738109b20 0x7f373819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373659c640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3738104270 0x7f373819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 -- 172.21.15.19:0/2997439787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3738104270 msgr2=0x7f373819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3738104270 0x7f373819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 -- 172.21.15.19:0/2997439787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3738109b20 msgr2=0x7f373819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3738109b20 0x7f373819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3736d9d640 1 -- 172.21.15.19:0/2997439787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373819f590 con 0x7f3738109fc0 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f373659c640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3738104270 0x7f373819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.889+0000 7f3735d9b640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3738109b20 0x7f373819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.890+0000 7f3736d9d640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f373819efd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f372800db20 tx=0x7f372800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.890+0000 7f37277fe640 1 -- 172.21.15.19:0/2997439787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728017070 con 0x7f3738109fc0 2024-04-25T11:00:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.890+0000 7f37277fe640 1 -- 172.21.15.19:0/2997439787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3728003e20 con 0x7f3738109fc0 2024-04-25T11:00:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.890+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373819f880 con 0x7f3738109fc0 2024-04-25T11:00:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.890+0000 7f37277fe640 1 -- 172.21.15.19:0/2997439787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728010460 con 0x7f3738109fc0 2024-04-25T11:00:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.890+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373819fd50 con 0x7f3738109fc0 2024-04-25T11:00:27.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 Processor -- start 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f37277fe640 1 -- 172.21.15.19:0/2997439787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3728003970 con 0x7f3738109fc0 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 -- start start 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f98454f390 0x55f9848f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9845264d0 0x55f9848f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 0x55f9848fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f984855d80 con 0x55f98454f390 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f98480fa20 con 0x55f9845264d0 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3743e53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f984851d40 con 0x55f98452e6b0 2024-04-25T11:00:27.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f98454f390 0x55f9848f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f98454f390 0x55f9848f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51906/0 (socket says 172.21.15.19:51906) 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3735d9b640 1 -- 172.21.15.19:0/65179352 learned_addr learned my addr 172.21.15.19:0/65179352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f37277fe640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f37200afc60 0x7f37200b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.893+0000 7f3735d9b640 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 0x55f9848fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f373659c640 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9845264d0 0x55f9848f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3735d9b640 1 -- 172.21.15.19:0/65179352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 msgr2=0x55f9848fd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3735d9b640 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 0x55f9848fd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3735d9b640 1 -- 172.21.15.19:0/65179352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9845264d0 msgr2=0x55f9848f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3735d9b640 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9845264d0 0x55f9848f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3735d9b640 1 -- 172.21.15.19:0/65179352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98481ac30 con 0x55f98454f390 2024-04-25T11:00:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3736d9d640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f37200afc60 0x7f37200b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f37277fe640 1 -- 172.21.15.19:0/2997439787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f37280dee90 con 0x7f3738109fc0 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3735d9b640 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f98454f390 0x55f9848f0810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f372c031440 tx=0x7f372c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467968caffb97a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3724ff9640 1 -- 172.21.15.19:0/65179352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c035050 con 0x55f98454f390 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3724ff9640 1 -- 172.21.15.19:0/65179352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c031780 con 0x55f98454f390 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.894+0000 7f3724ff9640 1 -- 172.21.15.19:0/65179352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c0046c0 con 0x55f98454f390 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 -- 172.21.15.19:0/65179352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f98454f390 msgr2=0x55f9848f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3736d9d640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f37200afc60 0x7f37200b2170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37280dc6c0 tx=0x7f3728004480 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f98454f390 0x55f9848f0810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f372c031440 tx=0x7f372c0058b0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 -- 172.21.15.19:0/65179352 shutdown_connections 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 0x55f9848fd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9845264d0 0x55f9848f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 --2- 172.21.15.19:0/65179352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f98454f390 0x55f9848f0810 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 -- 172.21.15.19:0/65179352 >> 172.21.15.19:0/65179352 conn(0x55f9845434e0 msgr2=0x55f9848ecfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 -- 172.21.15.19:0/65179352 shutdown_connections 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.895+0000 7f3743e53740 1 -- 172.21.15.19:0/65179352 wait complete. 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 -- start start 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 Processor -- start 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9848fb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 0x55f9849071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f9848fdd30 con 0x55f9845264d0 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9847dcdc0 con 0x55f98452e6b0 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f9848fdeb0 con 0x55f98454f390 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3743e53740 10 client.0 _ll_register_callbacks cb 0x55f9847f8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9848fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9848fb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51912/0 (socket says 172.21.15.19:51912) 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 -- 172.21.15.19:0/1577952849 learned_addr learned my addr 172.21.15.19:0/1577952849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f373659c640 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 -- 172.21.15.19:0/1577952849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 msgr2=0x55f9849071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 0x55f9849071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 -- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 msgr2=0x55f9848fb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f3735d9b640 1 -- 172.21.15.19:0/1577952849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9845673f0 con 0x55f9845264d0 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.896+0000 7f373659c640 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f3735d9b640 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9848fb270 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f372c006010 tx=0x7f372c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5ddd7e0f7d991f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f371dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f371dffb640 1 -- 172.21.15.19:0/1577952849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c04c020 con 0x55f9845264d0 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372c031780 con 0x55f9845264d0 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f371dffb640 1 -- 172.21.15.19:0/1577952849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f372c003800 con 0x55f9845264d0 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f371dffb640 1 -- 172.21.15.19:0/1577952849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f372c057050 con 0x55f9845264d0 2024-04-25T11:00:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f3743e53740 10 client.16458 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9846317d0 con 0x55f9845264d0 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f371dffb640 1 -- 172.21.15.19:0/1577952849 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c031780 con 0x55f9845264d0 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f371dffb640 1 -- 172.21.15.19:0/1577952849 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f372c03a780 con 0x55f9845264d0 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f3743e53740 10 client.16458 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.897+0000 7f3743e53740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.898+0000 7f3743e53740 20 client.16458 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.898+0000 7f3743e53740 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f98450df60 0x55f98490d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.898+0000 7f3743e53740 4 client.16458 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.898+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f984863990 con 0x55f98450df60 2024-04-25T11:00:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.898+0000 7f373659c640 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f98450df60 0x55f98490d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.898+0000 7f371dffb640 10 client.16458 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.898+0000 7f373659c640 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f98450df60 0x55f98490d490 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aac7e9bdaa8c6e2 server_cookie=9ddbd5fa200d7761 in_seq=0 out_seq=0 2024-04-25T11:00:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f371dffb640 1 -- 172.21.15.19:0/1577952849 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f984863990 con 0x55f98450df60 2024-04-25T11:00:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f371dffb640 10 client.16458 handle_command_reply: tid=0 2024-04-25T11:00:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 client.16458 shutdown 2024-04-25T11:00:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 msgr2=0x55f9848fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9848fb270 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f372c006010 tx=0x7f372c0045f0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 shutdown_connections 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f98450df60 0x55f98490d490 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 0x55f9849071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f371dffb640 0 client.16458 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 --2- 172.21.15.19:0/1577952849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9848fb270 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 >> 172.21.15.19:0/1577952849 conn(0x55f9845434e0 msgr2=0x55f9848f2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.899+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 shutdown_connections 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.900+0000 7f3743e53740 1 -- 172.21.15.19:0/1577952849 wait complete. 2024-04-25T11:00:27.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.900+0000 7f3743e53740 20 client.16458 trim_cache size 0 max 16384 2024-04-25T11:00:27.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.913+0000 7f3743e53740 1 Processor -- start 2024-04-25T11:00:27.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.913+0000 7f3743e53740 1 -- start start 2024-04-25T11:00:27.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.913+0000 7f3743e53740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9849071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.913+0000 7f3743e53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.913+0000 7f3743e53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 0x55f9848f5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.913+0000 7f3743e53740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37381ad7c0 con 0x55f9845264d0 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3743e53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37381b0040 con 0x55f98452e6b0 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3743e53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37381ade30 con 0x55f98454f390 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9849071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9849071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51914/0 (socket says 172.21.15.19:51914) 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 -- 172.21.15.19:0/2722906616 learned_addr learned my addr 172.21.15.19:0/2722906616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 -- 172.21.15.19:0/2722906616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 msgr2=0x55f9848f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 0x55f9848f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f373659c640 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 -- 172.21.15.19:0/2722906616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 msgr2=0x55f9848fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 -- 172.21.15.19:0/2722906616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9848f95e0 con 0x55f9845264d0 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f373659c640 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f3735d9b640 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9849071a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f372c031650 tx=0x7f372c004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19ec0cc34a792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f371ffff640 1 -- 172.21.15.19:0/2722906616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c04c040 con 0x55f9845264d0 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f371ffff640 1 -- 172.21.15.19:0/2722906616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c031700 con 0x55f9845264d0 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.914+0000 7f371ffff640 1 -- 172.21.15.19:0/2722906616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c035040 con 0x55f9845264d0 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 -- 172.21.15.19:0/2722906616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 msgr2=0x55f9849071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9849071a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f372c031650 tx=0x7f372c004490 comp rx=0 tx=0).stop 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 -- 172.21.15.19:0/2722906616 shutdown_connections 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98454f390 0x55f9848f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98452e6b0 0x55f9848fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 --2- 172.21.15.19:0/2722906616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f9849071a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 -- 172.21.15.19:0/2722906616 >> 172.21.15.19:0/2722906616 conn(0x55f9845434e0 msgr2=0x55f9848fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 -- 172.21.15.19:0/2722906616 shutdown_connections 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 -- 172.21.15.19:0/2722906616 wait complete. 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 -- start start 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.915+0000 7f3743e53740 1 Processor -- start 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3743e53740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f984910b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3743e53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 0x55f98490de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3743e53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98454f390 0x55f98490e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3743e53740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37381ae500 con 0x55f9845264d0 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3743e53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37381af970 con 0x55f98454f390 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3743e53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37381aebd0 con 0x55f98452e6b0 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3743e53740 10 client.0 _ll_register_callbacks cb 0x7f372c031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f984910b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f984910b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51922/0 (socket says 172.21.15.19:51922) 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 -- 172.21.15.19:0/1289414385 learned_addr learned my addr 172.21.15.19:0/1289414385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98454f390 0x55f98490e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 -- 172.21.15.19:0/1289414385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 msgr2=0x55f98490de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 0x55f98490de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 -- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98454f390 msgr2=0x55f98490e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98454f390 0x55f98490e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 -- 172.21.15.19:0/1289414385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f98490ebe0 con 0x55f9845264d0 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f3735d9b640 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f984910b20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f372c02a930 tx=0x7f372c0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd24c5da1de321ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f371dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f371dffb640 1 -- 172.21.15.19:0/1289414385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c035070 con 0x55f9845264d0 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.916+0000 7f371dffb640 1 -- 172.21.15.19:0/1289414385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f372c031c00 con 0x55f9845264d0 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372c031700 con 0x55f9845264d0 2024-04-25T11:00:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f371dffb640 1 -- 172.21.15.19:0/1289414385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f372c04c040 con 0x55f9845264d0 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 10 client.16464 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9849754f0 con 0x55f9845264d0 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f371dffb640 1 -- 172.21.15.19:0/1289414385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c031700 con 0x55f9845264d0 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f371dffb640 1 -- 172.21.15.19:0/1289414385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f372c0039c0 con 0x55f9845264d0 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 10 client.16464 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 20 client.16464 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9844e33a0 0x55f984977b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 4 client.16464 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f98485d800 con 0x55f9844e33a0 2024-04-25T11:00:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.917+0000 7f373659c640 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9844e33a0 0x55f984977b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f371dffb640 10 client.16464 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f373659c640 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9844e33a0 0x55f984977b90 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e175c0ac6daad1c server_cookie=32f87242de4a07f3 in_seq=0 out_seq=0 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f371dffb640 1 -- 172.21.15.19:0/1289414385 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f98485d800 con 0x55f9844e33a0 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f371dffb640 10 client.16464 handle_command_reply: tid=0 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 client.16464 shutdown 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 msgr2=0x55f984910b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f984910b20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f372c02a930 tx=0x7f372c0054d0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 shutdown_connections 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f9844e33a0 0x55f984977b90 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f98454f390 0x55f98490e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f371dffb640 0 client.16464 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f98452e6b0 0x55f98490de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 --2- 172.21.15.19:0/1289414385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9845264d0 0x55f984910b20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 >> 172.21.15.19:0/1289414385 conn(0x55f9845434e0 msgr2=0x55f9848fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 shutdown_connections 2024-04-25T11:00:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 1 -- 172.21.15.19:0/1289414385 wait complete. 2024-04-25T11:00:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.918+0000 7f3743e53740 20 client.16464 trim_cache size 0 max 16384 2024-04-25T11:00:27.928 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:00:27.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f37200afc60 msgr2=0x7f37200b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f37200afc60 0x7f37200b2170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37280dc6c0 tx=0x7f3728004480 comp rx=0 tx=0).stop 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 msgr2=0x7f373819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f373819efd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f372800db20 tx=0x7f372800b260 comp rx=0 tx=0).stop 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 shutdown_connections 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f37200afc60 0x7f37200b2170 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738109fc0 0x7f373819efd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3738109b20 0x7f373819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 --2- 172.21.15.19:0/2997439787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3738104270 0x7f373819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 >> 172.21.15.19:0/2997439787 conn(0x7f37380ff1e0 msgr2=0x7f373810f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 shutdown_connections 2024-04-25T11:00:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:27.919+0000 7f373cfa8640 1 -- 172.21.15.19:0/2997439787 wait complete. 2024-04-25T11:00:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:28 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/390787934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:00:28.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:28 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/390787934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:00:28.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:28 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/390787934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:00:28.932 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:00:28.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:28.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:00:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 -- 172.21.15.19:0/1473328259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44109b20 msgr2=0x7f6d441036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 --2- 172.21.15.19:0/1473328259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44109b20 0x7f6d441036c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009900 tx=0x7f6d3402a410 comp rx=0 tx=0).stop 2024-04-25T11:00:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 -- 172.21.15.19:0/1473328259 shutdown_connections 2024-04-25T11:00:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 --2- 172.21.15.19:0/1473328259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d44104270 0x7f6d44110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 --2- 172.21.15.19:0/1473328259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d44109fc0 0x7f6d44103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 --2- 172.21.15.19:0/1473328259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44109b20 0x7f6d441036c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 -- 172.21.15.19:0/1473328259 >> 172.21.15.19:0/1473328259 conn(0x7f6d440ff1e0 msgr2=0x7f6d44101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 -- 172.21.15.19:0/1473328259 shutdown_connections 2024-04-25T11:00:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.063+0000 7f6d49b96640 1 -- 172.21.15.19:0/1473328259 wait complete. 2024-04-25T11:00:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 Processor -- start 2024-04-25T11:00:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 -- start start 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44104270 0x7f6d441a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d44109b20 0x7f6d441a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d44109fc0 0x7f6d4419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113d70 con 0x7f6d44104270 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113bf0 con 0x7f6d44109b20 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d49b96640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113ef0 con 0x7f6d44109fc0 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44104270 0x7f6d441a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44104270 0x7f6d441a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51954/0 (socket says 172.21.15.19:51954) 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.064+0000 7f6d437fe640 1 -- 172.21.15.19:0/707663991 learned_addr learned my addr 172.21.15.19:0/707663991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d43fff640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d44109fc0 0x7f6d4419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d44109b20 0x7f6d441a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d437fe640 1 -- 172.21.15.19:0/707663991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d44109fc0 msgr2=0x7f6d4419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d437fe640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d44109fc0 0x7f6d4419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d437fe640 1 -- 172.21.15.19:0/707663991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d44109b20 msgr2=0x7f6d441a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d437fe640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d44109b20 0x7f6d441a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d437fe640 1 -- 172.21.15.19:0/707663991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4419ada0 con 0x7f6d44104270 2024-04-25T11:00:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d43fff640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d44109fc0 0x7f6d4419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d437fe640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44104270 0x7f6d441a04f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6d34032fd0 tx=0x7f6d34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d40ff9640 1 -- 172.21.15.19:0/707663991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3403f030 con 0x7f6d44104270 2024-04-25T11:00:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d40ff9640 1 -- 172.21.15.19:0/707663991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d34004490 con 0x7f6d44104270 2024-04-25T11:00:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4419afd0 con 0x7f6d44104270 2024-04-25T11:00:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d40ff9640 1 -- 172.21.15.19:0/707663991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34035070 con 0x7f6d44104270 2024-04-25T11:00:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.065+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4419b4c0 con 0x7f6d44104270 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d40ff9640 1 -- 172.21.15.19:0/707663991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6d34042030 con 0x7f6d44104270 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 Processor -- start 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 -- start start 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b268de620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b268df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559b26844b00 con 0x559b2651d730 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b267e0920 con 0x559b26515550 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559b26840ac0 con 0x559b2653e410 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d42ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b268df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d42ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b268df0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51964/0 (socket says 172.21.15.19:51964) 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.068+0000 7f6d42ffd640 1 -- 172.21.15.19:0/2119067972 learned_addr learned my addr 172.21.15.19:0/2119067972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d43fff640 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b268de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d40ff9640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6d2c0afc60 0x7f6d2c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d437fe640 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d43fff640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6d2c0afc60 0x7f6d2c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d42ffd640 1 -- 172.21.15.19:0/2119067972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 msgr2=0x559b268de620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b268de620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d42ffd640 1 -- 172.21.15.19:0/2119067972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 msgr2=0x559b268deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268deb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d42ffd640 1 -- 172.21.15.19:0/2119067972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2661e7e0 con 0x559b2651d730 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d437fe640 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d40ff9640 1 -- 172.21.15.19:0/707663991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6d340ff570 con 0x7f6d44104270 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b268df0a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b760 tx=0x7f6d3800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377039e7f413313a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d29ffb640 1 -- 172.21.15.19:0/2119067972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800d950 con 0x559b2651d730 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d43fff640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6d2c0afc60 0x7f6d2c0b2170 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x559b268db790 tx=0x7f6d3c004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d29ffb640 1 -- 172.21.15.19:0/2119067972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38019070 con 0x559b2651d730 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.069+0000 7f6d29ffb640 1 -- 172.21.15.19:0/2119067972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38014da0 con 0x559b2651d730 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 -- 172.21.15.19:0/2119067972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 msgr2=0x559b268df0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b268df0a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b760 tx=0x7f6d3800bc20 comp rx=0 tx=0).stop 2024-04-25T11:00:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 -- 172.21.15.19:0/2119067972 shutdown_connections 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b268df0a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/2119067972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b268de620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 -- 172.21.15.19:0/2119067972 >> 172.21.15.19:0/2119067972 conn(0x559b26532560 msgr2=0x559b268dadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 -- 172.21.15.19:0/2119067972 shutdown_connections 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.070+0000 7f6d50a4a740 1 -- 172.21.15.19:0/2119067972 wait complete. 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 -- start start 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 Processor -- start 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b26515550 0x559b268e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2653e410 0x559b268f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559b268eb970 con 0x559b26515550 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b267de5d0 con 0x559b2653e410 2024-04-25T11:00:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559b268ebaf0 con 0x559b2651d730 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d50a4a740 10 client.0 _ll_register_callbacks cb 0x559b267cc080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b26515550 0x559b268e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b26515550 0x559b268e8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51968/0 (socket says 172.21.15.19:51968) 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d437fe640 1 -- 172.21.15.19:0/3155554743 learned_addr learned my addr 172.21.15.19:0/3155554743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.071+0000 7f6d43fff640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2653e410 0x559b268f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d437fe640 1 -- 172.21.15.19:0/3155554743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 msgr2=0x559b268e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d437fe640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d437fe640 1 -- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2653e410 msgr2=0x559b268f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d437fe640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2653e410 0x559b268f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d437fe640 1 -- 172.21.15.19:0/3155554743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2667ef70 con 0x559b26515550 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d43fff640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2653e410 0x559b268f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d437fe640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b26515550 0x559b268e8b60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6d340fd2f0 tx=0x7f6d3403b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726f108ee995b3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3155554743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34049030 con 0x559b26515550 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d38019070 con 0x559b26515550 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3155554743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6d340c7ba0 con 0x559b26515550 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3155554743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6d340c7090 con 0x559b26515550 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d50a4a740 10 client.16476 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559b26516170 con 0x559b26515550 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.072+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3155554743 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38019070 con 0x559b26515550 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3155554743 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f6d340d0110 con 0x559b26515550 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d50a4a740 10 client.16476 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d50a4a740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d50a4a740 20 client.16476 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264fd1f0 0x559b268faf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d50a4a740 4 client.16476 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b26852710 con 0x559b264fd1f0 2024-04-25T11:00:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264fd1f0 0x559b268faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264fd1f0 0x559b268faf80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947c5b3b0ea3f3e8 server_cookie=767e055b3d2d20bd in_seq=0 out_seq=0 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.073+0000 7f6d22ffd640 10 client.16476 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.074+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3155554743 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b26852710 con 0x559b264fd1f0 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.074+0000 7f6d22ffd640 10 client.16476 handle_command_reply: tid=0 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.074+0000 7f6d50a4a740 1 client.16476 shutdown 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b26515550 msgr2=0x559b268e8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b26515550 0x559b268e8b60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6d340fd2f0 tx=0x7f6d3403b640 comp rx=0 tx=0).stop 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 shutdown_connections 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264fd1f0 0x559b268faf80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2653e410 0x559b268f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3155554743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b26515550 0x559b268e8b60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d22ffd640 0 client.16476 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 >> 172.21.15.19:0/3155554743 conn(0x559b26532560 msgr2=0x559b268e0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 shutdown_connections 2024-04-25T11:00:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3155554743 wait complete. 2024-04-25T11:00:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.075+0000 7f6d50a4a740 20 client.16476 trim_cache size 0 max 16384 2024-04-25T11:00:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 Processor -- start 2024-04-25T11:00:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 -- start start 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268eaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b26900740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b26900c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441ad920 con 0x559b2651d730 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441b01a0 con 0x559b26515550 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441adf90 con 0x559b2653e410 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.093+0000 7f6d42ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b26900740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b26900740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51972/0 (socket says 172.21.15.19:51972) 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 -- 172.21.15.19:0/1717790817 learned_addr learned my addr 172.21.15.19:0/1717790817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d43fff640 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b26900c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d437fe640 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268eaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 -- 172.21.15.19:0/1717790817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 msgr2=0x559b26900c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b26900c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 -- 172.21.15.19:0/1717790817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 msgr2=0x559b268eaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268eaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 -- 172.21.15.19:0/1717790817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b268de3b0 con 0x559b2651d730 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d437fe640 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268eaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b26900740 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800c010 tx=0x7f6d380149a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354d79e30dc51e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d297fa640 1 -- 172.21.15.19:0/1717790817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38023070 con 0x559b2651d730 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d297fa640 1 -- 172.21.15.19:0/1717790817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38019040 con 0x559b2651d730 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.094+0000 7f6d297fa640 1 -- 172.21.15.19:0/1717790817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38023570 con 0x559b2651d730 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 -- 172.21.15.19:0/1717790817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 msgr2=0x559b26900740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b26900740 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800c010 tx=0x7f6d380149a0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 -- 172.21.15.19:0/1717790817 shutdown_connections 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2653e410 0x559b26900c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2651d730 0x559b26900740 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/1717790817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268eaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 -- 172.21.15.19:0/1717790817 >> 172.21.15.19:0/1717790817 conn(0x559b26532560 msgr2=0x559b268d9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 -- 172.21.15.19:0/1717790817 shutdown_connections 2024-04-25T11:00:29.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 -- 172.21.15.19:0/1717790817 wait complete. 2024-04-25T11:00:29.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.095+0000 7f6d50a4a740 1 -- start start 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 1 Processor -- start 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268ed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2653e410 0x559b268ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441ae660 con 0x559b2653e410 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441afad0 con 0x559b26515550 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441aed30 con 0x559b2651d730 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d50a4a740 10 client.0 _ll_register_callbacks cb 0x7f6d340fd180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2653e410 0x559b268ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2653e410 0x559b268ed560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51988/0 (socket says 172.21.15.19:51988) 2024-04-25T11:00:29.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 -- 172.21.15.19:0/3121833717 learned_addr learned my addr 172.21.15.19:0/3121833717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d437fe640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 -- 172.21.15.19:0/3121833717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 msgr2=0x559b268ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268ed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 -- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 msgr2=0x559b268ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 -- 172.21.15.19:0/3121833717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b268df5b0 con 0x559b2653e410 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d437fe640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d42ffd640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.096+0000 7f6d43fff640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2653e410 0x559b268ed560 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c013440 tx=0x7f6d3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d75a36a56d6003 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3121833717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c019040 con 0x559b2653e410 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3121833717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6d3c012d80 con 0x559b2653e410 2024-04-25T11:00:29.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d38019040 con 0x559b2653e410 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3121833717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c011e60 con 0x559b2653e410 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 10 client.16482 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559b269634a0 con 0x559b2653e410 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3121833717 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38019040 con 0x559b2653e410 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3121833717 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f6d3c01f030 con 0x559b2653e410 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 10 client.16482 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 20 client.16482 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264d2630 0x559b26965c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 4 client.16482 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.097+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559b268db250 con 0x559b264d2630 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.098+0000 7f6d437fe640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264d2630 0x559b26965c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.098+0000 7f6d437fe640 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264d2630 0x559b26965c90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc0cf0f43b33657 server_cookie=c2840e29cd6f718 in_seq=0 out_seq=0 2024-04-25T11:00:29.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.098+0000 7f6d22ffd640 10 client.16482 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d22ffd640 1 -- 172.21.15.19:0/3121833717 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559b268db250 con 0x559b264d2630 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d22ffd640 10 client.16482 handle_command_reply: tid=0 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 client.16482 shutdown 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2653e410 msgr2=0x559b268ed560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2653e410 0x559b268ed560 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c013440 tx=0x7f6d3c004a40 comp rx=0 tx=0).stop 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 shutdown_connections 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559b264d2630 0x559b26965c90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b2653e410 0x559b268ed560 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b2651d730 0x559b268ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d22ffd640 0 client.16482 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 --2- 172.21.15.19:0/3121833717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b26515550 0x559b268ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 >> 172.21.15.19:0/3121833717 conn(0x559b26532560 msgr2=0x559b268d9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 shutdown_connections 2024-04-25T11:00:29.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.099+0000 7f6d50a4a740 1 -- 172.21.15.19:0/3121833717 wait complete. 2024-04-25T11:00:29.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d50a4a740 20 client.16482 trim_cache size 0 max 16384 2024-04-25T11:00:29.106 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:00:29.106 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:00:29.106 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "2e3f7012-494f-4c34-9331-66719cc9a16c", 2024-04-25T11:00:29.106 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:00:29.106 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6d2c0afc60 msgr2=0x7f6d2c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6d2c0afc60 0x7f6d2c0b2170 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x559b268db790 tx=0x7f6d3c004ab0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44104270 msgr2=0x7f6d441a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44104270 0x7f6d441a04f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6d34032fd0 tx=0x7f6d34004270 comp rx=0 tx=0).stop 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 shutdown_connections 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6d2c0afc60 0x7f6d2c0b2170 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d44109fc0 0x7f6d4419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d44109b20 0x7f6d441a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 --2- 172.21.15.19:0/707663991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d44104270 0x7f6d441a04f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 >> 172.21.15.19:0/707663991 conn(0x7f6d440ff1e0 msgr2=0x7f6d440ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 shutdown_connections 2024-04-25T11:00:29.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.100+0000 7f6d49b96640 1 -- 172.21.15.19:0/707663991 wait complete. 2024-04-25T11:00:29.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:00:29.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:00:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.224+0000 7f92f84d8640 1 -- 172.21.15.19:0/508148257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 msgr2=0x7f92f0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.224+0000 7f92f84d8640 1 --2- 172.21.15.19:0/508148257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f0110630 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009ca0 tx=0x7f92e002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 -- 172.21.15.19:0/508148257 shutdown_connections 2024-04-25T11:00:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 --2- 172.21.15.19:0/508148257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f0110630 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 --2- 172.21.15.19:0/508148257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f0109fb0 0x7f92f0103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 --2- 172.21.15.19:0/508148257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f0109b10 0x7f92f01036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 -- 172.21.15.19:0/508148257 >> 172.21.15.19:0/508148257 conn(0x7f92f00ff210 msgr2=0x7f92f0101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 -- 172.21.15.19:0/508148257 shutdown_connections 2024-04-25T11:00:29.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 -- 172.21.15.19:0/508148257 wait complete. 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 Processor -- start 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.225+0000 7f92f84d8640 1 -- start start 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f84d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f84d8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f0109b10 0x7f92f019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f84d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f0109fb0 0x7f92f019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f84d8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113d00 con 0x7f92f0104260 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f84d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113b80 con 0x7f92f0109fb0 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f84d8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113e80 con 0x7f92f0109b10 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52010/0 (socket says 172.21.15.19:52010) 2024-04-25T11:00:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 -- 172.21.15.19:0/3549536315 learned_addr learned my addr 172.21.15.19:0/3549536315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f5a73640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f0109b10 0x7f92f019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6a75640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f0109fb0 0x7f92f019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 -- 172.21.15.19:0/3549536315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f0109b10 msgr2=0x7f92f019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f0109b10 0x7f92f019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 -- 172.21.15.19:0/3549536315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f0109fb0 msgr2=0x7f92f019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f0109fb0 0x7f92f019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 -- 172.21.15.19:0/3549536315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f019f520 con 0x7f92f0104260 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f5a73640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f0109b10 0x7f92f019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92f6274640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f019a740 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f92e800d950 tx=0x7f92e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.226+0000 7f92ef7fe640 1 -- 172.21.15.19:0/3549536315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8017070 con 0x7f92f0104260 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.227+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f019f810 con 0x7f92f0104260 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.227+0000 7f92ef7fe640 1 -- 172.21.15.19:0/3549536315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8009c30 con 0x7f92f0104260 2024-04-25T11:00:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.227+0000 7f92ef7fe640 1 -- 172.21.15.19:0/3549536315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8003d20 con 0x7f92f0104260 2024-04-25T11:00:29.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.227+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a7210 con 0x7f92f0104260 2024-04-25T11:00:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.229+0000 7f92ff385740 1 Processor -- start 2024-04-25T11:00:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.229+0000 7f92ff385740 1 -- start start 2024-04-25T11:00:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ef7fe640 1 -- 172.21.15.19:0/3549536315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f92e8003ec0 con 0x7f92f0104260 2024-04-25T11:00:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ff385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a271a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ff385740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a271fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ff385740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a27e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ff385740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d7a1d7d80 con 0x559d79ea84d0 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ff385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d7a1918b0 con 0x559d79ed1390 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ff385740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559d7a1d3d40 con 0x559d79eb06b0 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a271fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a271fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52014/0 (socket says 172.21.15.19:52014) 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 -- 172.21.15.19:0/1741221551 learned_addr learned my addr 172.21.15.19:0/1741221551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6a75640 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a271a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f5a73640 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a27e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 -- 172.21.15.19:0/1741221551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 msgr2=0x559d7a27e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a27e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 -- 172.21.15.19:0/1741221551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 msgr2=0x559d7a271a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a271a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 -- 172.21.15.19:0/1741221551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7a162a40 con 0x559d79ea84d0 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92f6274640 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a271fa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f92e80dc410 tx=0x7f92e80e57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38302fdce6ce1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ef7fe640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92d80afc60 0x7f92d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.230+0000 7f92ecbfe640 1 -- 172.21.15.19:0/1741221551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80a6690 con 0x559d79ea84d0 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.231+0000 7f92f6a75640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92d80afc60 0x7f92d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.231+0000 7f92ecbfe640 1 -- 172.21.15.19:0/1741221551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8009de0 con 0x559d79ea84d0 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.231+0000 7f92ecbfe640 1 -- 172.21.15.19:0/1741221551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80a6870 con 0x559d79ea84d0 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.231+0000 7f92ef7fe640 1 -- 172.21.15.19:0/3549536315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f92e80dedb0 con 0x7f92f0104260 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 -- 172.21.15.19:0/1741221551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 msgr2=0x559d7a271fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92f6a75640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92d80afc60 0x7f92d80b2170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92e00094e0 tx=0x7f92e002f3a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a271fa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f92e80dc410 tx=0x7f92e80e57f0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 -- 172.21.15.19:0/1741221551 shutdown_connections 2024-04-25T11:00:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a27e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a271fa0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 --2- 172.21.15.19:0/1741221551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a271a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 -- 172.21.15.19:0/1741221551 >> 172.21.15.19:0/1741221551 conn(0x559d79ec54e0 msgr2=0x559d7a26e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 -- 172.21.15.19:0/1741221551 shutdown_connections 2024-04-25T11:00:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 -- 172.21.15.19:0/1741221551 wait complete. 2024-04-25T11:00:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 -- start start 2024-04-25T11:00:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.232+0000 7f92ff385740 1 Processor -- start 2024-04-25T11:00:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a280dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d7a27efe0 con 0x559d79ea84d0 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d7a15ed10 con 0x559d79ed1390 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559d7a27f160 con 0x559d79eb06b0 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 10 client.0 _ll_register_callbacks cb 0x559d7a18b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2869d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52018/0 (socket says 172.21.15.19:52018) 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 -- 172.21.15.19:0/2407608653 learned_addr learned my addr 172.21.15.19:0/2407608653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f5a73640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 -- 172.21.15.19:0/2407608653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 msgr2=0x559d7a286f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6a75640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a280dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 -- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 msgr2=0x559d7a280dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a280dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 -- 172.21.15.19:0/2407608653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d79fb37d0 con 0x559d79ea84d0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f5a73640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6a75640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a280dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92f6274640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2869d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009610 tx=0x7f92e80dc440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272f005de0c69f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92d5ffb640 1 -- 172.21.15.19:0/2407608653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80a6a60 con 0x559d79ea84d0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92d5ffb640 1 -- 172.21.15.19:0/2407608653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f92e80c9040 con 0x559d79ea84d0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e8009de0 con 0x559d79ea84d0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.233+0000 7f92d5ffb640 1 -- 172.21.15.19:0/2407608653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f92e80a6cf0 con 0x559d79ea84d0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 10 client.16494 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559d79ee93f0 con 0x559d79ea84d0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92d5ffb640 1 -- 172.21.15.19:0/2407608653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8009de0 con 0x559d79ea84d0 2024-04-25T11:00:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92d5ffb640 1 -- 172.21.15.19:0/2407608653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f92e80a31b0 con 0x559d79ea84d0 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 10 client.16494 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 20 client.16494 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e8ff60 0x559d7a292900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 4 client.16494 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d7a1e5990 con 0x559d79e8ff60 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.234+0000 7f92f5a73640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e8ff60 0x559d7a292900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.235+0000 7f92f5a73640 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e8ff60 0x559d7a292900 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7780f70b8b0275 server_cookie=acf613b0eb3802c5 in_seq=0 out_seq=0 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.235+0000 7f92d5ffb640 10 client.16494 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.235+0000 7f92d5ffb640 1 -- 172.21.15.19:0/2407608653 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d7a1e5990 con 0x559d79e8ff60 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.235+0000 7f92d5ffb640 10 client.16494 handle_command_reply: tid=0 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.235+0000 7f92ff385740 1 client.16494 shutdown 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.235+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 msgr2=0x559d7a2869d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.235+0000 7f92ff385740 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2869d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009610 tx=0x7f92e80dc440 comp rx=0 tx=0).stop 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 shutdown_connections 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e8ff60 0x559d7a292900 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a280dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 --2- 172.21.15.19:0/2407608653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2869d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92d5ffb640 0 client.16494 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 >> 172.21.15.19:0/2407608653 conn(0x559d79ec54e0 msgr2=0x559d7a273780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 shutdown_connections 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 1 -- 172.21.15.19:0/2407608653 wait complete. 2024-04-25T11:00:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.236+0000 7f92ff385740 20 client.16494 trim_cache size 0 max 16384 2024-04-25T11:00:29.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 Processor -- start 2024-04-25T11:00:29.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 -- start start 2024-04-25T11:00:29.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a26d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ab7c0 con 0x559d79ea84d0 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ae060 con 0x559d79ed1390 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92ff385740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01abe50 con 0x559d79eb06b0 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92f6274640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2800c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52020/0 (socket says 172.21.15.19:52020) 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92f6274640 1 -- 172.21.15.19:0/1356867184 learned_addr learned my addr 172.21.15.19:0/1356867184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92f6274640 1 -- 172.21.15.19:0/1356867184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 msgr2=0x559d7a286c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92f6274640 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92f6274640 1 -- 172.21.15.19:0/1356867184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 msgr2=0x559d7a26d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.254+0000 7f92f6a75640 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a26d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.255+0000 7f92f6274640 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a26d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.255+0000 7f92f6274640 1 -- 172.21.15.19:0/1356867184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d79ea90f0 con 0x559d79ea84d0 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.255+0000 7f92f6a75640 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a26d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.255+0000 7f92f6274640 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2800c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f92e8005fd0 tx=0x7f92e80dc5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d42415a9c9b44b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.255+0000 7f92d7fff640 1 -- 172.21.15.19:0/1356867184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80a2890 con 0x559d79ea84d0 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.255+0000 7f92d7fff640 1 -- 172.21.15.19:0/1356867184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e80a6a30 con 0x559d79ea84d0 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.255+0000 7f92d7fff640 1 -- 172.21.15.19:0/1356867184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80a2c10 con 0x559d79ea84d0 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- 172.21.15.19:0/1356867184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 msgr2=0x559d7a2800c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2800c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f92e8005fd0 tx=0x7f92e80dc5b0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- 172.21.15.19:0/1356867184 shutdown_connections 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ed1390 0x559d7a26d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79eb06b0 0x559d7a286c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 --2- 172.21.15.19:0/1356867184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79ea84d0 0x559d7a2800c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- 172.21.15.19:0/1356867184 >> 172.21.15.19:0/1356867184 conn(0x559d79ec54e0 msgr2=0x559d7a286500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- 172.21.15.19:0/1356867184 shutdown_connections 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- 172.21.15.19:0/1356867184 wait complete. 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- start start 2024-04-25T11:00:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 Processor -- start 2024-04-25T11:00:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ea84d0 0x559d7a28ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79eb06b0 0x559d7a28bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79ed1390 0x559d7a2f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ac520 con 0x559d79eb06b0 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ad990 con 0x559d79ea84d0 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01acbf0 con 0x559d79ed1390 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92ff385740 10 client.0 _ll_register_callbacks cb 0x7f92e80e2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.256+0000 7f92f5a73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79eb06b0 0x559d7a28bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79eb06b0 0x559d7a28bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52032/0 (socket says 172.21.15.19:52032) 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 -- 172.21.15.19:0/3168336110 learned_addr learned my addr 172.21.15.19:0/3168336110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f6a75640 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79ed1390 0x559d7a2f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f6274640 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ea84d0 0x559d7a28ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 -- 172.21.15.19:0/3168336110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79ed1390 msgr2=0x559d7a2f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79ed1390 0x559d7a2f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 -- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ea84d0 msgr2=0x559d7a28ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ea84d0 0x559d7a28ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 -- 172.21.15.19:0/3168336110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7a1984b0 con 0x559d79eb06b0 2024-04-25T11:00:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92f5a73640 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79eb06b0 0x559d7a28bfd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f92f00ff660 tx=0x7f92e4013b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfd16c019cd7382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92d5ffb640 1 -- 172.21.15.19:0/3168336110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4011b70 con 0x559d79eb06b0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92d5ffb640 1 -- 172.21.15.19:0/3168336110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f92e4011d10 con 0x559d79eb06b0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e80a6a30 con 0x559d79eb06b0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92d5ffb640 1 -- 172.21.15.19:0/3168336110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f92e402b050 con 0x559d79eb06b0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92ff385740 10 client.16500 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.257+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559d7a2f9df0 con 0x559d79eb06b0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92d5ffb640 1 -- 172.21.15.19:0/3168336110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e80a6a30 con 0x559d79eb06b0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92d5ffb640 1 -- 172.21.15.19:0/3168336110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f92e4010690 con 0x559d79eb06b0 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92ff385740 10 client.16500 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92ff385740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92ff385740 20 client.16500 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92ff385740 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e653a0 0x559d7a2fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92ff385740 4 client.16500 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d7a1df800 con 0x559d79e653a0 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.258+0000 7f92f6274640 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e653a0 0x559d7a2fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92d5ffb640 10 client.16500 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92f6274640 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e653a0 0x559d7a2fc5e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c63d07e48a1eff server_cookie=f2d74d865eec079c in_seq=0 out_seq=0 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92d5ffb640 1 -- 172.21.15.19:0/3168336110 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559d7a1df800 con 0x559d79e653a0 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92d5ffb640 10 client.16500 handle_command_reply: tid=0 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 client.16500 shutdown 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79eb06b0 msgr2=0x559d7a28bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79eb06b0 0x559d7a28bfd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f92f00ff660 tx=0x7f92e4013b20 comp rx=0 tx=0).stop 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 shutdown_connections 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559d79e653a0 0x559d7a2fc5e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d79ed1390 0x559d7a2f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d79eb06b0 0x559d7a28bfd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 --2- 172.21.15.19:0/3168336110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d79ea84d0 0x559d7a28ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 >> 172.21.15.19:0/3168336110 conn(0x559d79ec54e0 msgr2=0x559d7a27fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92d5ffb640 0 client.16500 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 shutdown_connections 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 1 -- 172.21.15.19:0/3168336110 wait complete. 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.259+0000 7f92ff385740 20 client.16500 trim_cache size 0 max 16384 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "2e3f7012-494f-4c34-9331-66719cc9a16c": { 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:00:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "2e3f7012-494f-4c34-9331-66719cc9a16c", 2024-04-25T11:00:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:00:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:00:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:00:29.267 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92d80afc60 msgr2=0x7f92d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92d80afc60 0x7f92d80b2170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92e00094e0 tx=0x7f92e002f3a0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 msgr2=0x7f92f019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f019a740 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f92e800d950 tx=0x7f92e800de10 comp rx=0 tx=0).stop 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 shutdown_connections 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f92d80afc60 0x7f92d80b2170 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f0109fb0 0x7f92f019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f0109b10 0x7f92f019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 --2- 172.21.15.19:0/3549536315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0104260 0x7f92f019a740 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 >> 172.21.15.19:0/3549536315 conn(0x7f92f00ff210 msgr2=0x7f92f0106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 shutdown_connections 2024-04-25T11:00:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:29.260+0000 7f92f84d8640 1 -- 172.21.15.19:0/3549536315 wait complete. 2024-04-25T11:00:29.271 INFO:tasks.cephfs.filesystem:scrub status for tag:2e3f7012-494f-4c34-9331-66719cc9a16c - {'path': '/', 'tag': '2e3f7012-494f-4c34-9331-66719cc9a16c', 'options': 'recursive,force'} 2024-04-25T11:00:29.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:29 smithi019 ceph-mon[29209]: pgmap v182: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:29.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:29 smithi169 ceph-mon[33049]: pgmap v182: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:29.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:29 smithi112 ceph-mon[32799]: pgmap v182: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:30.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:30 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:00:30.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:30 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:00:30.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:30 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:00:30.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:30 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:00:30.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:30 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:00:30.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:30 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:00:30.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:30 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:00:30.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:30 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:00:30.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:30 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:00:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:30 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:00:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:30 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:00:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:30 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:00:31.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:31 smithi169 ceph-mon[33049]: pgmap v183: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:31.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:31 smithi112 ceph-mon[32799]: pgmap v183: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:31.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:31 smithi019 ceph-mon[29209]: pgmap v183: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:32.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:32 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:00:32.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:32 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:00:32.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:32 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:00:33.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:33 smithi169 ceph-mon[33049]: pgmap v184: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:33.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:33 smithi112 ceph-mon[32799]: pgmap v184: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:33.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:33 smithi019 ceph-mon[29209]: pgmap v184: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:35.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:35 smithi169 ceph-mon[33049]: pgmap v185: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:35.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:35 smithi112 ceph-mon[32799]: pgmap v185: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:35.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:35 smithi019 ceph-mon[29209]: pgmap v185: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:37.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:37 smithi169 ceph-mon[33049]: pgmap v186: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:37.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:37 smithi112 ceph-mon[32799]: pgmap v186: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:37.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:37 smithi019 ceph-mon[29209]: pgmap v186: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:39.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:39 smithi169 ceph-mon[33049]: pgmap v187: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:39.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:39 smithi112 ceph-mon[32799]: pgmap v187: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:39.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:39 smithi019 ceph-mon[29209]: pgmap v187: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:41 smithi169 ceph-mon[33049]: pgmap v188: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:41 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:00:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:41 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:00:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:41 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:00:41.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:41 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:00:41.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:41 smithi112 ceph-mon[32799]: pgmap v188: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:41.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:41 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:00:41.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:41 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:00:41.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:41 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:00:41.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:41 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:00:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:41 smithi019 ceph-mon[29209]: pgmap v188: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:00:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:41 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:00:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:41 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:00:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:41 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:00:41.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:41 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:00:43.754 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:43 smithi169 ceph-mon[33049]: pgmap v189: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:43.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:43 smithi019 ceph-mon[29209]: pgmap v189: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:43.973 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:43 smithi112 ceph-mon[32799]: pgmap v189: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:00:45.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:45 smithi169 ceph-mon[33049]: pgmap v190: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:45.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:45 smithi112 ceph-mon[32799]: pgmap v190: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:45.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:45 smithi019 ceph-mon[29209]: pgmap v190: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:46.810 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:46 smithi112 ceph-mon[32799]: pgmap v191: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:46.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:46 smithi019 ceph-mon[29209]: pgmap v191: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:47.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:46 smithi169 ceph-mon[33049]: pgmap v191: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:49.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:49 smithi019 ceph-mon[29209]: pgmap v192: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:49.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:49 smithi169 ceph-mon[33049]: pgmap v192: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:49.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:49 smithi112 ceph-mon[32799]: pgmap v192: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:51.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:51 smithi019 ceph-mon[29209]: pgmap v193: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:51 smithi169 ceph-mon[33049]: pgmap v193: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:51.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:51 smithi112 ceph-mon[32799]: pgmap v193: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:53 smithi019 ceph-mon[29209]: pgmap v194: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:53 smithi169 ceph-mon[33049]: pgmap v194: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:53.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:53 smithi112 ceph-mon[32799]: pgmap v194: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:55.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:55 smithi019 ceph-mon[29209]: pgmap v195: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:55.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:55 smithi169 ceph-mon[33049]: pgmap v195: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:55.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:55 smithi112 ceph-mon[32799]: pgmap v195: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:57.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:57 smithi019 ceph-mon[29209]: pgmap v196: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:57 smithi169 ceph-mon[33049]: pgmap v196: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:57.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:57 smithi112 ceph-mon[32799]: pgmap v196: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:00:59.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:00:59.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:00:59.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:00:59 smithi019 ceph-mon[29209]: pgmap v197: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:59.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 -- 172.21.15.19:0/3349908267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc108a20 msgr2=0x7fb3dc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/3349908267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc108a20 0x7fb3dc1025b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009900 tx=0x7fb3cc02a410 comp rx=0 tx=0).stop 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 -- 172.21.15.19:0/3349908267 shutdown_connections 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/3349908267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3dc10d380 0x7fb3dc10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/3349908267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3dc102af0 0x7fb3dc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/3349908267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc108a20 0x7fb3dc1025b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 -- 172.21.15.19:0/3349908267 >> 172.21.15.19:0/3349908267 conn(0x7fb3dc078100 msgr2=0x7fb3dc078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 -- 172.21.15.19:0/3349908267 shutdown_connections 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.403+0000 7fb3e2b14640 1 -- 172.21.15.19:0/3349908267 wait complete. 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 Processor -- start 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 -- start start 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3dc102af0 0x7fb3dc1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3dc108a20 0x7fb3dc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc10d380 0x7fb3dc19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc112be0 con 0x7fb3dc10d380 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc112a60 con 0x7fb3dc108a20 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e2b14640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc112d60 con 0x7fb3dc102af0 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e10b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc10d380 0x7fb3dc19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e10b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc10d380 0x7fb3dc19de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37956/0 (socket says 172.21.15.19:37956) 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e10b1640 1 -- 172.21.15.19:0/2455700018 learned_addr learned my addr 172.21.15.19:0/2455700018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3dc102af0 0x7fb3dc1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.404+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3dc108a20 0x7fb3dc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e10b1640 1 -- 172.21.15.19:0/2455700018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3dc102af0 msgr2=0x7fb3dc1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e10b1640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3dc102af0 0x7fb3dc1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e10b1640 1 -- 172.21.15.19:0/2455700018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3dc108a20 msgr2=0x7fb3dc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e10b1640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3dc108a20 0x7fb3dc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e10b1640 1 -- 172.21.15.19:0/2455700018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc19e440 con 0x7fb3dc10d380 2024-04-25T11:00:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e10b1640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc10d380 0x7fb3dc19de00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400ebb0 tx=0x7fb3d4007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3d9ffb640 1 -- 172.21.15.19:0/2455700018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4007ca0 con 0x7fb3dc10d380 2024-04-25T11:00:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3d9ffb640 1 -- 172.21.15.19:0/2455700018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d4007e40 con 0x7fb3dc10d380 2024-04-25T11:00:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc19e6d0 con 0x7fb3dc10d380 2024-04-25T11:00:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3d9ffb640 1 -- 172.21.15.19:0/2455700018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40057d0 con 0x7fb3dc10d380 2024-04-25T11:00:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.405+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc19ebd0 con 0x7fb3dc10d380 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3d9ffb640 1 -- 172.21.15.19:0/2455700018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb3d4004a40 con 0x7fb3dc10d380 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 Processor -- start 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 -- start start 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0ed390 0x5561ca48da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0c44d0 0x5561ca48dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0cc6b0 0x5561ca49a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561ca3f3d80 con 0x5561ca0ed390 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561ca3ad8b0 con 0x5561ca0cc6b0 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.408+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5561ca3efd40 con 0x5561ca0c44d0 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0ed390 0x5561ca48da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3d9ffb640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3c40afc60 0x7fb3c40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0ed390 0x5561ca48da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37958/0 (socket says 172.21.15.19:37958) 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 -- 172.21.15.19:0/3863356141 learned_addr learned my addr 172.21.15.19:0/3863356141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0c44d0 0x5561ca48dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3e10b1640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3c40afc60 0x7fb3c40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0cc6b0 0x5561ca49a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3d9ffb640 1 -- 172.21.15.19:0/2455700018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb3d40e04c0 con 0x7fb3dc10d380 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 -- 172.21.15.19:0/3863356141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0c44d0 msgr2=0x5561ca48dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0c44d0 0x5561ca48dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 -- 172.21.15.19:0/3863356141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0cc6b0 msgr2=0x5561ca49a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0cc6b0 0x5561ca49a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 -- 172.21.15.19:0/3863356141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ca37ea40 con 0x5561ca0ed390 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0ed390 0x5561ca48da60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000bc00 tx=0x7fb3d00041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fef3953257efdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3e10b1640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3c40afc60 0x7fb3c40b2170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40ddcf0 tx=0x7fb3d4091040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3c2ffd640 1 -- 172.21.15.19:0/3863356141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0010460 con 0x5561ca0ed390 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3c2ffd640 1 -- 172.21.15.19:0/3863356141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d0010ab0 con 0x5561ca0ed390 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.409+0000 7fb3c2ffd640 1 -- 172.21.15.19:0/3863356141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000f8b0 con 0x5561ca0ed390 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 -- 172.21.15.19:0/3863356141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0ed390 msgr2=0x5561ca48da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0ed390 0x5561ca48da60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000bc00 tx=0x7fb3d00041d0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 -- 172.21.15.19:0/3863356141 shutdown_connections 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0cc6b0 0x5561ca49a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0c44d0 0x5561ca48dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/3863356141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0ed390 0x5561ca48da60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 -- 172.21.15.19:0/3863356141 >> 172.21.15.19:0/3863356141 conn(0x5561ca0e14e0 msgr2=0x5561ca48a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 -- 172.21.15.19:0/3863356141 shutdown_connections 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.410+0000 7fb3e99c9740 1 -- 172.21.15.19:0/3863356141 wait complete. 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 -- start start 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 Processor -- start 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca4985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca498b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 0x5561ca49e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561ca49af80 con 0x5561ca0cc6b0 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561ca37ad10 con 0x5561ca0c44d0 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5561ca49b100 con 0x5561ca0ed390 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e99c9740 10 client.0 _ll_register_callbacks cb 0x5561ca3a7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.411+0000 7fb3e08b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca498b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca498b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37968/0 (socket says 172.21.15.19:37968) 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 -- 172.21.15.19:0/2926605061 learned_addr learned my addr 172.21.15.19:0/2926605061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 0x5561ca49e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca4985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 -- 172.21.15.19:0/2926605061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 msgr2=0x5561ca49e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 0x5561ca49e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 -- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 msgr2=0x5561ca4985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca4985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 -- 172.21.15.19:0/2926605061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ca1cf7d0 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca498b30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc033000 tx=0x7fb3cc032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd18407d521a925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3bbfff640 1 -- 172.21.15.19:0/2926605061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc035040 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3bbfff640 1 -- 172.21.15.19:0/2926605061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb3cc004050 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d0010ab0 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3bbfff640 1 -- 172.21.15.19:0/2926605061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb3cc004250 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e99c9740 10 client.16512 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.412+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5561ca1053f0 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3bbfff640 1 -- 172.21.15.19:0/2926605061 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d0010ab0 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3bbfff640 1 -- 172.21.15.19:0/2926605061 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb3cc0049f0 con 0x5561ca0cc6b0 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3e99c9740 10 client.16512 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3e99c9740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3e99c9740 20 client.16512 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0abf60 0x5561ca4aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3e99c9740 4 client.16512 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ca401990 con 0x5561ca0abf60 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.413+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0abf60 0x5561ca4aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.414+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0abf60 0x5561ca4aa5a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952b3cfdbeb7fffa server_cookie=672cbd9137f1b0fd in_seq=0 out_seq=0 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.414+0000 7fb3bbfff640 10 client.16512 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.414+0000 7fb3bbfff640 1 -- 172.21.15.19:0/2926605061 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561ca401990 con 0x5561ca0abf60 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.414+0000 7fb3bbfff640 10 client.16512 handle_command_reply: tid=0 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.414+0000 7fb3e99c9740 1 client.16512 shutdown 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 msgr2=0x5561ca498b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca498b30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc033000 tx=0x7fb3cc032f60 comp rx=0 tx=0).stop 2024-04-25T11:00:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 shutdown_connections 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0abf60 0x5561ca4aa5a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 0x5561ca49e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca498b30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3bbfff640 0 client.16512 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/2926605061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca4985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 >> 172.21.15.19:0/2926605061 conn(0x5561ca0e14e0 msgr2=0x5561ca48f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 shutdown_connections 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 1 -- 172.21.15.19:0/2926605061 wait complete. 2024-04-25T11:00:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.415+0000 7fb3e99c9740 20 client.16512 trim_cache size 0 max 16384 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.433+0000 7fb3e99c9740 1 Processor -- start 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.433+0000 7fb3e99c9740 1 -- start start 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca49a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca49e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 0x5561ca49e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1ab600 con 0x5561ca0cc6b0 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1ade80 con 0x5561ca0c44d0 2024-04-25T11:00:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1abc70 con 0x5561ca0ed390 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca49e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca49e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37978/0 (socket says 172.21.15.19:37978) 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 -- 172.21.15.19:0/1547133213 learned_addr learned my addr 172.21.15.19:0/1547133213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca49a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 -- 172.21.15.19:0/1547133213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 msgr2=0x5561ca49e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 0x5561ca49e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 -- 172.21.15.19:0/1547133213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 msgr2=0x5561ca49a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca49a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 -- 172.21.15.19:0/1547133213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ca0c50f0 con 0x5561ca0cc6b0 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca49e3c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000c00 tx=0x7fb3cc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae686de7da79788 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3c27fc640 1 -- 172.21.15.19:0/1547133213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc035030 con 0x5561ca0cc6b0 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3c27fc640 1 -- 172.21.15.19:0/1547133213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3cc041070 con 0x5561ca0cc6b0 2024-04-25T11:00:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.434+0000 7fb3c27fc640 1 -- 172.21.15.19:0/1547133213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc003a40 con 0x5561ca0cc6b0 2024-04-25T11:00:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1547133213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 msgr2=0x5561ca49e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca49e3c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000c00 tx=0x7fb3cc0058d0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1547133213 shutdown_connections 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0ed390 0x5561ca49e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0cc6b0 0x5561ca49e3c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1547133213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0c44d0 0x5561ca49a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1547133213 >> 172.21.15.19:0/1547133213 conn(0x5561ca0e14e0 msgr2=0x5561ca497340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1547133213 shutdown_connections 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.435+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1547133213 wait complete. 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 -- start start 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 Processor -- start 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0c44d0 0x5561ca49fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0cc6b0 0x5561ca4a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0ed390 0x5561ca510900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1ac340 con 0x5561ca0c44d0 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1ad7b0 con 0x5561ca0ed390 2024-04-25T11:00:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1aca10 con 0x5561ca0cc6b0 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e99c9740 10 client.0 _ll_register_callbacks cb 0x7fb3cc005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0c44d0 0x5561ca49fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0c44d0 0x5561ca49fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37984/0 (socket says 172.21.15.19:37984) 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3dbfff640 1 -- 172.21.15.19:0/1236614353 learned_addr learned my addr 172.21.15.19:0/1236614353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.436+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0cc6b0 0x5561ca4a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0ed390 0x5561ca510900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3dbfff640 1 -- 172.21.15.19:0/1236614353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0cc6b0 msgr2=0x5561ca4a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0cc6b0 0x5561ca4a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3dbfff640 1 -- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0ed390 msgr2=0x5561ca510900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0ed390 0x5561ca510900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3dbfff640 1 -- 172.21.15.19:0/1236614353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ca3b44b0 con 0x5561ca0c44d0 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0cc6b0 0x5561ca4a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3dbfff640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0c44d0 0x5561ca49fbf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c010 tx=0x7fb3d000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146d62816107148a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3bbfff640 1 -- 172.21.15.19:0/1236614353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0028030 con 0x5561ca0c44d0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3bbfff640 1 -- 172.21.15.19:0/1236614353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fb3d0010660 con 0x5561ca0c44d0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3cc041070 con 0x5561ca0c44d0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3bbfff640 1 -- 172.21.15.19:0/1236614353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb3d002b040 con 0x5561ca0c44d0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3e99c9740 10 client.16518 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.437+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5561ca511ed0 con 0x5561ca0c44d0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3bbfff640 1 -- 172.21.15.19:0/1236614353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3cc041070 con 0x5561ca0c44d0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3bbfff640 1 -- 172.21.15.19:0/1236614353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb3d00045f0 con 0x5561ca0c44d0 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e99c9740 10 client.16518 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e99c9740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e99c9740 20 client.16518 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0813a0 0x5561ca5143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e99c9740 4 client.16518 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561ca48e330 con 0x5561ca0813a0 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0813a0 0x5561ca5143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3bbfff640 10 client.16518 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.438+0000 7fb3e08b0640 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0813a0 0x5561ca5143b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1646967a902ae9 server_cookie=26b1e77fb8e3d458 in_seq=0 out_seq=0 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3bbfff640 1 -- 172.21.15.19:0/1236614353 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561ca48e330 con 0x5561ca0813a0 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3bbfff640 10 client.16518 handle_command_reply: tid=0 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 client.16518 shutdown 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0c44d0 msgr2=0x5561ca49fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0c44d0 0x5561ca49fbf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c010 tx=0x7fb3d000b410 comp rx=0 tx=0).stop 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 shutdown_connections 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5561ca0813a0 0x5561ca5143b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561ca0ed390 0x5561ca510900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3bbfff640 0 client.16518 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561ca0cc6b0 0x5561ca4a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 --2- 172.21.15.19:0/1236614353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ca0c44d0 0x5561ca49fbf0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 >> 172.21.15.19:0/1236614353 conn(0x5561ca0e14e0 msgr2=0x5561ca497340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 shutdown_connections 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 1 -- 172.21.15.19:0/1236614353 wait complete. 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.439+0000 7fb3e99c9740 20 client.16518 trim_cache size 0 max 16384 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:00:59.447 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:00:59.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3c40afc60 msgr2=0x7fb3c40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3c40afc60 0x7fb3c40b2170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40ddcf0 tx=0x7fb3d4091040 comp rx=0 tx=0).stop 2024-04-25T11:00:59.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc10d380 msgr2=0x7fb3dc19de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc10d380 0x7fb3dc19de00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400ebb0 tx=0x7fb3d4007590 comp rx=0 tx=0).stop 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 shutdown_connections 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb3c40afc60 0x7fb3c40b2170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3dc10d380 0x7fb3dc19de00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3dc108a20 0x7fb3dc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 --2- 172.21.15.19:0/2455700018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3dc102af0 0x7fb3dc1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 >> 172.21.15.19:0/2455700018 conn(0x7fb3dc078100 msgr2=0x7fb3dc1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 shutdown_connections 2024-04-25T11:00:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.440+0000 7fb3e2b14640 1 -- 172.21.15.19:0/2455700018 wait complete. 2024-04-25T11:00:59.453 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:00:59.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:59.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:00:59 smithi169 ceph-mon[33049]: pgmap v197: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:59.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:00:59 smithi112 ceph-mon[32799]: pgmap v197: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:00:59.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.563+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/3487090928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 msgr2=0x7fbd68102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.563+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/3487090928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd68102620 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009900 tx=0x7fbd5802a410 comp rx=0 tx=0).stop 2024-04-25T11:00:59.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.563+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/3487090928 shutdown_connections 2024-04-25T11:00:59.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.563+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/3487090928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd6810d3a0 0x7fbd6810f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.563+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/3487090928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd68102b60 0x7fbd68102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.563+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/3487090928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd68102620 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.563+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/3487090928 >> 172.21.15.19:0/3487090928 conn(0x7fbd68078040 msgr2=0x7fbd68078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.564+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/3487090928 shutdown_connections 2024-04-25T11:00:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.564+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/3487090928 wait complete. 2024-04-25T11:00:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.564+0000 7fbd6dcd8640 1 Processor -- start 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.564+0000 7fbd6dcd8640 1 -- start start 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd68102b60 0x7fbd6819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd6819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6810d3a0 0x7fbd6819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd68112c00 con 0x7fbd68108a40 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd68112a80 con 0x7fbd6810d3a0 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd68112d80 con 0x7fbd68102b60 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd6819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd6819ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38000/0 (socket says 172.21.15.19:38000) 2024-04-25T11:00:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 -- 172.21.15.19:0/4049252566 learned_addr learned my addr 172.21.15.19:0/4049252566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd67fff640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6810d3a0 0x7fbd6819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 -- 172.21.15.19:0/4049252566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd68102b60 msgr2=0x7fbd6819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd68102b60 0x7fbd6819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 -- 172.21.15.19:0/4049252566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6810d3a0 msgr2=0x7fbd6819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6810d3a0 0x7fbd6819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 -- 172.21.15.19:0/4049252566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd681a6850 con 0x7fbd68108a40 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd66ffd640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd6819ba80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbd50005fd0 tx=0x7fbd5000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd64ff9640 1 -- 172.21.15.19:0/4049252566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50017070 con 0x7fbd68108a40 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd681a6b40 con 0x7fbd68108a40 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd64ff9640 1 -- 172.21.15.19:0/4049252566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd50009c30 con 0x7fbd68108a40 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd64ff9640 1 -- 172.21.15.19:0/4049252566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50003d10 con 0x7fbd68108a40 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.565+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd681a70e0 con 0x7fbd68108a40 2024-04-25T11:00:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.566+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6806a190 con 0x7fbd68108a40 2024-04-25T11:00:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.568+0000 7fbd64ff9640 1 -- 172.21.15.19:0/4049252566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fbd50003eb0 con 0x7fbd68108a40 2024-04-25T11:00:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.568+0000 7fbd64ff9640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbd540afc60 0x7fbd540b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.569+0000 7fbd677fe640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbd540afc60 0x7fbd540b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.569+0000 7fbd64ff9640 1 -- 172.21.15.19:0/4049252566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fbd500df4d0 con 0x7fbd68108a40 2024-04-25T11:00:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.569+0000 7fbd677fe640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbd540afc60 0x7fbd540b2170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd5802a930 tx=0x7fbd58031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.572+0000 7fbd64ff9640 1 -- 172.21.15.19:0/4049252566 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd50020030 con 0x7fbd68108a40 2024-04-25T11:00:59.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.803+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd680753c0 con 0x7fbd68108a40 2024-04-25T11:00:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.804+0000 7fbd64ff9640 1 -- 172.21.15.19:0/4049252566 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fbd50009de0 con 0x7fbd68108a40 2024-04-25T11:00:59.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:00:59.806 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:00:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:00:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbd540afc60 msgr2=0x7fbd540b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbd540afc60 0x7fbd540b2170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd5802a930 tx=0x7fbd58031400 comp rx=0 tx=0).stop 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 msgr2=0x7fbd6819ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd6819ba80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbd50005fd0 tx=0x7fbd5000dde0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 shutdown_connections 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbd540afc60 0x7fbd540b2170 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6810d3a0 0x7fbd6819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd68108a40 0x7fbd6819ba80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 --2- 172.21.15.19:0/4049252566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd68102b60 0x7fbd6819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 >> 172.21.15.19:0/4049252566 conn(0x7fbd68078040 msgr2=0x7fbd681040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 shutdown_connections 2024-04-25T11:00:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.806+0000 7fbd6dcd8640 1 -- 172.21.15.19:0/4049252566 wait complete. 2024-04-25T11:00:59.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:00:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.932+0000 7fd4e417d640 1 -- 172.21.15.19:0/428616843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc109b30 msgr2=0x7fd4dc103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.932+0000 7fd4e417d640 1 --2- 172.21.15.19:0/428616843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc109b30 0x7fd4dc103720 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009900 tx=0x7fd4d802a410 comp rx=0 tx=0).stop 2024-04-25T11:00:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.932+0000 7fd4e417d640 1 -- 172.21.15.19:0/428616843 shutdown_connections 2024-04-25T11:00:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.932+0000 7fd4e417d640 1 --2- 172.21.15.19:0/428616843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc1042d0 0x7fd4dc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.932+0000 7fd4e417d640 1 --2- 172.21.15.19:0/428616843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc109fd0 0x7fd4dc103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.932+0000 7fd4e417d640 1 --2- 172.21.15.19:0/428616843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc109b30 0x7fd4dc103720 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.932+0000 7fd4e417d640 1 -- 172.21.15.19:0/428616843 >> 172.21.15.19:0/428616843 conn(0x7fd4dc079280 msgr2=0x7fd4dc0796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 -- 172.21.15.19:0/428616843 shutdown_connections 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 -- 172.21.15.19:0/428616843 wait complete. 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 Processor -- start 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 -- start start 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc1042d0 0x7fd4dc19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc109b30 0x7fd4dc19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc109fd0 0x7fd4dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc113d20 con 0x7fd4dc1042d0 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc113ba0 con 0x7fd4dc109b30 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.933+0000 7fd4e417d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc113ea0 con 0x7fd4dc109fd0 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc1042d0 0x7fd4dc19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc1042d0 0x7fd4dc19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38018/0 (socket says 172.21.15.19:38018) 2024-04-25T11:00:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 -- 172.21.15.19:0/4193574020 learned_addr learned my addr 172.21.15.19:0/4193574020 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e271a640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc109fd0 0x7fd4dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1718640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc109b30 0x7fd4dc19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 -- 172.21.15.19:0/4193574020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc109fd0 msgr2=0x7fd4dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc109fd0 0x7fd4dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 -- 172.21.15.19:0/4193574020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc109b30 msgr2=0x7fd4dc19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc109b30 0x7fd4dc19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 -- 172.21.15.19:0/4193574020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc19f580 con 0x7fd4dc1042d0 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc1042d0 0x7fd4dc19a720 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006010 tx=0x7fd4d8004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/4193574020 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d803f030 con 0x7fd4dc1042d0 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/4193574020 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d8003aa0 con 0x7fd4dc1042d0 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc19f810 con 0x7fd4dc1042d0 2024-04-25T11:00:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/4193574020 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8035070 con 0x7fd4dc1042d0 2024-04-25T11:00:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.934+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc19fd00 con 0x7fd4dc1042d0 2024-04-25T11:00:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.937+0000 7fd4eb02d740 1 Processor -- start 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.937+0000 7fd4eb02d740 1 -- start start 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.937+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/4193574020 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd4d8042030 con 0x7fd4dc1042d0 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b83390 0x559e54f24810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f24d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e54e89d80 con 0x559e54b5a4d0 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e54e43a20 con 0x559e54b626b0 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559e54e85d40 con 0x559e54b83390 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f24d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f24d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38034/0 (socket says 172.21.15.19:38034) 2024-04-25T11:00:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 -- 172.21.15.19:0/2727045052 learned_addr learned my addr 172.21.15.19:0/2727045052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1718640 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e271a640 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b83390 0x559e54f24810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4d2ffd640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd4c80afc60 0x7fd4c80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 -- 172.21.15.19:0/2727045052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b83390 msgr2=0x559e54f24810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b83390 0x559e54f24810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 -- 172.21.15.19:0/2727045052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 msgr2=0x559e54f316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 -- 172.21.15.19:0/2727045052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e54e4ec30 con 0x559e54b5a4d0 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e271a640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd4c80afc60 0x7fd4c80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/4193574020 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd4d80ff530 con 0x7fd4dc1042d0 2024-04-25T11:00:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.938+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f24d50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80fcc20 tx=0x7fd4d80c35c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fb169899263f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.939+0000 7fd4c7fff640 1 -- 172.21.15.19:0/2727045052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8049070 con 0x559e54b5a4d0 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.939+0000 7fd4c7fff640 1 -- 172.21.15.19:0/2727045052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d803b460 con 0x559e54b5a4d0 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.939+0000 7fd4c7fff640 1 -- 172.21.15.19:0/2727045052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80c7050 con 0x559e54b5a4d0 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.939+0000 7fd4eb02d740 1 -- 172.21.15.19:0/2727045052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 msgr2=0x559e54f24d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.939+0000 7fd4e271a640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd4c80afc60 0x7fd4c80b2170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x559e54f21980 tx=0x7fd4d4004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.939+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f24d50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80fcc20 tx=0x7fd4d80c35c0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 -- 172.21.15.19:0/2727045052 shutdown_connections 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f24d50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/2727045052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b83390 0x559e54f24810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 -- 172.21.15.19:0/2727045052 >> 172.21.15.19:0/2727045052 conn(0x559e54b774e0 msgr2=0x559e54f20fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 -- 172.21.15.19:0/2727045052 shutdown_connections 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 -- 172.21.15.19:0/2727045052 wait complete. 2024-04-25T11:00:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 -- start start 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.940+0000 7fd4eb02d740 1 Processor -- start 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b5a4d0 0x559e54f2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f2f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b83390 0x559e54f3b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e54f31be0 con 0x559e54b83390 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e54e10dc0 con 0x559e54b626b0 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559e54f31d60 con 0x559e54b5a4d0 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4eb02d740 10 client.0 _ll_register_callbacks cb 0x559e54e2c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b83390 0x559e54f3b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b83390 0x559e54f3b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38048/0 (socket says 172.21.15.19:38048) 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 -- 172.21.15.19:0/1982545518 learned_addr learned my addr 172.21.15.19:0/1982545518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e1718640 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f2f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b5a4d0 0x559e54f2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 -- 172.21.15.19:0/1982545518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b5a4d0 msgr2=0x559e54f2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b5a4d0 0x559e54f2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 -- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 msgr2=0x559e54f2f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f2f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 -- 172.21.15.19:0/1982545518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e54b9b3f0 con 0x559e54b83390 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4e271a640 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b83390 0x559e54f3b010 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4003e50 tx=0x7fd4d4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a4105ab7cd5b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.941+0000 7fd4c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1982545518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4010960 con 0x559e54b83390 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e54e97990 con 0x559e54b83390 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1982545518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd4d4010b00 con 0x559e54b83390 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1982545518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd4d4010d00 con 0x559e54b83390 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 10 client.16536 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:59.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559e54c657d0 con 0x559e54b83390 2024-04-25T11:00:59.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1982545518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e54e97990 con 0x559e54b83390 2024-04-25T11:00:59.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1982545518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd4d40114d0 con 0x559e54b83390 2024-04-25T11:00:59.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 10 client.16536 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:59.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:59.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 20 client.16536 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:59.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b41f60 0x559e54f41400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 4 client.16536 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.942+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e54e91800 con 0x559e54b41f60 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.943+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b41f60 0x559e54f41400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.943+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b41f60 0x559e54f41400 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35111dbd992b4b02 server_cookie=5e7006239f8799e0 in_seq=0 out_seq=0 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.943+0000 7fd4c57fa640 10 client.16536 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1982545518 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e54e91800 con 0x559e54b41f60 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4c57fa640 10 client.16536 handle_command_reply: tid=0 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 client.16536 shutdown 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b83390 msgr2=0x559e54f3b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b83390 0x559e54f3b010 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4003e50 tx=0x7fd4d4004ab0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 shutdown_connections 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b41f60 0x559e54f41400 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b83390 0x559e54f3b010 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f2f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4c57fa640 0 client.16536 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1982545518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b5a4d0 0x559e54f2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 >> 172.21.15.19:0/1982545518 conn(0x559e54b774e0 msgr2=0x559e54f26530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 shutdown_connections 2024-04-25T11:00:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1982545518 wait complete. 2024-04-25T11:00:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.944+0000 7fd4eb02d740 20 client.16536 trim_cache size 0 max 16384 2024-04-25T11:00:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 Processor -- start 2024-04-25T11:00:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 -- start start 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f46bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1ad970 con 0x559e54b5a4d0 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1b01f0 con 0x559e54b626b0 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1adfe0 con 0x559e54b41f60 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4e1718640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4e1718640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f3ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38056/0 (socket says 172.21.15.19:38056) 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4e1718640 1 -- 172.21.15.19:0/3161384946 learned_addr learned my addr 172.21.15.19:0/3161384946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.958+0000 7fd4e271a640 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f46bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4e1718640 1 -- 172.21.15.19:0/3161384946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 msgr2=0x559e54f2e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4e1718640 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f2e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4e1718640 1 -- 172.21.15.19:0/3161384946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 msgr2=0x559e54f46bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4e1718640 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f46bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4e1718640 1 -- 172.21.15.19:0/3161384946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e54f3bb10 con 0x559e54b5a4d0 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4e1718640 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f3ac30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc003720 tx=0x7fd4cc003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70778519abe3b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4c77fe640 1 -- 172.21.15.19:0/3161384946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00cc30 con 0x559e54b5a4d0 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4c77fe640 1 -- 172.21.15.19:0/3161384946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc00cdd0 con 0x559e54b5a4d0 2024-04-25T11:00:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.959+0000 7fd4c77fe640 1 -- 172.21.15.19:0/3161384946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc01a6d0 con 0x559e54b5a4d0 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 -- 172.21.15.19:0/3161384946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 msgr2=0x559e54f3ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f3ac30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc003720 tx=0x7fd4cc003be0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 -- 172.21.15.19:0/3161384946 shutdown_connections 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b626b0 0x559e54f46bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b5a4d0 0x559e54f3ac30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/3161384946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f2e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 -- 172.21.15.19:0/3161384946 >> 172.21.15.19:0/3161384946 conn(0x559e54b774e0 msgr2=0x559e54f306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 -- 172.21.15.19:0/3161384946 shutdown_connections 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 -- 172.21.15.19:0/3161384946 wait complete. 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 -- start start 2024-04-25T11:00:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.960+0000 7fd4eb02d740 1 Processor -- start 2024-04-25T11:00:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f32cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b5a4d0 0x559e54f33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4eb02d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b626b0 0x559e54f33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1ae6b0 con 0x559e54b626b0 2024-04-25T11:00:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1afb20 con 0x559e54b5a4d0 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4eb02d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1aed80 con 0x559e54b41f60 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4eb02d740 10 client.0 _ll_register_callbacks cb 0x7fd4d4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b626b0 0x559e54f33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b626b0 0x559e54f33740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38064/0 (socket says 172.21.15.19:38064) 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 -- 172.21.15.19:0/1465521088 learned_addr learned my addr 172.21.15.19:0/1465521088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f32cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e1718640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b5a4d0 0x559e54f33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 -- 172.21.15.19:0/1465521088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 msgr2=0x559e54f32cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f32cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 -- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b5a4d0 msgr2=0x559e54f33200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b5a4d0 0x559e54f33200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 -- 172.21.15.19:0/1465521088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e54fa8840 con 0x559e54b626b0 2024-04-25T11:00:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e1718640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b5a4d0 0x559e54f33200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4e271a640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b626b0 0x559e54f33740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4003e50 tx=0x7fd4d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134f660436282043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.961+0000 7fd4c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1465521088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4003a00 con 0x559e54b626b0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1465521088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fd4d4011440 con 0x559e54b626b0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4cc00cdd0 con 0x559e54b626b0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1465521088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd4d4011640 con 0x559e54b626b0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 10 client.16542 fetch_fsmap learned FSMap version 20 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559e54fa9aa0 con 0x559e54b626b0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1465521088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc00cdd0 con 0x559e54b626b0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1465521088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd4d4031650 con 0x559e54b626b0 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 10 client.16542 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 20 client.16542 populate_metadata read hostname 'smithi019' 2024-04-25T11:00:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b83390 0x559e54fac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 4 client.16542 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e54e0c610 con 0x559e54b83390 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.962+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b83390 0x559e54fac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4c57fa640 10 client.16542 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4e1f19640 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b83390 0x559e54fac1c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50bb0b2f0ccf392 server_cookie=7d9c467ec74126cd in_seq=0 out_seq=0 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4c57fa640 1 -- 172.21.15.19:0/1465521088 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e54e0c610 con 0x559e54b83390 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4c57fa640 10 client.16542 handle_command_reply: tid=0 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 client.16542 shutdown 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b626b0 msgr2=0x559e54f33740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b626b0 0x559e54f33740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4003e50 tx=0x7fd4d4016000 comp rx=0 tx=0).stop 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 shutdown_connections 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x559e54b83390 0x559e54fac1c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e54b626b0 0x559e54f33740 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e54b5a4d0 0x559e54f33200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 --2- 172.21.15.19:0/1465521088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e54b41f60 0x559e54f32cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4c57fa640 0 client.16542 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:00:59.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 >> 172.21.15.19:0/1465521088 conn(0x559e54b774e0 msgr2=0x559e54f2e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 shutdown_connections 2024-04-25T11:00:59.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 1 -- 172.21.15.19:0/1465521088 wait complete. 2024-04-25T11:00:59.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.963+0000 7fd4eb02d740 20 client.16542 trim_cache size 0 max 16384 2024-04-25T11:00:59.973 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:00:59.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd4c80afc60 msgr2=0x7fd4c80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd4c80afc60 0x7fd4c80b2170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x559e54f21980 tx=0x7fd4d4004a40 comp rx=0 tx=0).stop 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc1042d0 msgr2=0x7fd4dc19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc1042d0 0x7fd4dc19a720 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006010 tx=0x7fd4d8004060 comp rx=0 tx=0).stop 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 shutdown_connections 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fd4c80afc60 0x7fd4c80b2170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc109fd0 0x7fd4dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc109b30 0x7fd4dc19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 --2- 172.21.15.19:0/4193574020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4dc1042d0 0x7fd4dc19a720 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 >> 172.21.15.19:0/4193574020 conn(0x7fd4dc079280 msgr2=0x7fd4dc1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 shutdown_connections 2024-04-25T11:00:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:00:59.964+0000 7fd4e417d640 1 -- 172.21.15.19:0/4193574020 wait complete. 2024-04-25T11:01:00.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:00 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4049252566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:01:00.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:00 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4049252566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:01:00.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:00 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4049252566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:01:00.977 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:01:00.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:01:00.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 -- 172.21.15.19:0/1996652909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098104270 msgr2=0x7fa098110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/1996652909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098104270 0x7fa098110640 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa090009ca0 tx=0x7fa09002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 -- 172.21.15.19:0/1996652909 shutdown_connections 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/1996652909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098104270 0x7fa098110640 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/1996652909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa098109fc0 0x7fa098103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/1996652909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098109b20 0x7fa0981036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 -- 172.21.15.19:0/1996652909 >> 172.21.15.19:0/1996652909 conn(0x7fa0980ff1e0 msgr2=0x7fa098101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 -- 172.21.15.19:0/1996652909 shutdown_connections 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 -- 172.21.15.19:0/1996652909 wait complete. 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.099+0000 7fa09e1a8640 1 Processor -- start 2024-04-25T11:01:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa09e1a8640 1 -- start start 2024-04-25T11:01:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa09e1a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa098104270 0x7fa09819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa09e1a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098109b20 0x7fa09819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa09e1a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098109fc0 0x7fa09819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa09e1a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa098113d10 con 0x7fa098109b20 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa09e1a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa098113b90 con 0x7fa098109fc0 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa09e1a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa098113e90 con 0x7fa098104270 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098109b20 0x7fa09819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098109b20 0x7fa09819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38092/0 (socket says 172.21.15.19:38092) 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 -- 172.21.15.19:0/3900146979 learned_addr learned my addr 172.21.15.19:0/3900146979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa097fff640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098109fc0 0x7fa09819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa0977fe640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa098104270 0x7fa09819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 -- 172.21.15.19:0/3900146979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa098104270 msgr2=0x7fa09819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa098104270 0x7fa09819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 -- 172.21.15.19:0/3900146979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098109fc0 msgr2=0x7fa09819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098109fc0 0x7fa09819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.100+0000 7fa096ffd640 1 -- 172.21.15.19:0/3900146979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09819f4d0 con 0x7fa098109b20 2024-04-25T11:01:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.101+0000 7fa096ffd640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098109b20 0x7fa09819ac50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00d950 tx=0x7fa08c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.101+0000 7fa094ff9640 1 -- 172.21.15.19:0/3900146979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c017070 con 0x7fa098109b20 2024-04-25T11:01:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.101+0000 7fa094ff9640 1 -- 172.21.15.19:0/3900146979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c009c30 con 0x7fa098109b20 2024-04-25T11:01:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.101+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09819f7c0 con 0x7fa098109b20 2024-04-25T11:01:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.101+0000 7fa094ff9640 1 -- 172.21.15.19:0/3900146979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c003d10 con 0x7fa098109b20 2024-04-25T11:01:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.101+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09819fd10 con 0x7fa098109b20 2024-04-25T11:01:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.103+0000 7fa094ff9640 1 -- 172.21.15.19:0/3900146979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa08c003eb0 con 0x7fa098109b20 2024-04-25T11:01:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 Processor -- start 2024-04-25T11:01:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 -- start start 2024-04-25T11:01:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df17f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56267de7db00 con 0x56267db77410 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56267de19920 con 0x56267db56730 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0a5057740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56267de79ac0 con 0x56267db4e550 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df174b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38096/0 (socket says 172.21.15.19:38096) 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa097fff640 1 -- 172.21.15.19:0/1021261045 learned_addr learned my addr 172.21.15.19:0/1021261045 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa0977fe640 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa094ff9640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa0800afc60 0x7fa0800b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa096ffd640 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df17f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.104+0000 7fa097fff640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa0800afc60 0x7fa0800b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa097fff640 1 -- 172.21.15.19:0/1021261045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 msgr2=0x56267df179f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa097fff640 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df179f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa097fff640 1 -- 172.21.15.19:0/1021261045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 msgr2=0x56267df17f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa097fff640 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df17f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa094ff9640 1 -- 172.21.15.19:0/3900146979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa08c0decc0 con 0x7fa098109b20 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa097fff640 1 -- 172.21.15.19:0/1021261045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267dc577e0 con 0x56267db77410 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa0977fe640 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa097fff640 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df174b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa090031f10 tx=0x7fa090031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4afbbf42822eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa07dffb640 1 -- 172.21.15.19:0/1021261045 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090003d40 con 0x56267db77410 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa07dffb640 1 -- 172.21.15.19:0/1021261045 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09004e070 con 0x56267db77410 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.105+0000 7fa07dffb640 1 -- 172.21.15.19:0/1021261045 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09002f950 con 0x56267db77410 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 -- 172.21.15.19:0/1021261045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 msgr2=0x56267df174b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df174b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa090031f10 tx=0x7fa090031f40 comp rx=0 tx=0).stop 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa097fff640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa0800afc60 0x7fa0800b2170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa09004d000 tx=0x7fa09004b000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 -- 172.21.15.19:0/1021261045 shutdown_connections 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df17f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 --2- 172.21.15.19:0/1021261045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df174b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 -- 172.21.15.19:0/1021261045 >> 172.21.15.19:0/1021261045 conn(0x56267db6b560 msgr2=0x56267df13dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 -- 172.21.15.19:0/1021261045 shutdown_connections 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 -- 172.21.15.19:0/1021261045 wait complete. 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.106+0000 7fa0a5057740 1 -- start start 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 1 Processor -- start 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df260b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56267df28d80 con 0x56267db77410 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56267de175d0 con 0x56267db56730 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56267df28f00 con 0x56267db4e550 2024-04-25T11:01:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0a5057740 10 client.0 _ll_register_callbacks cb 0x56267de05080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38100/0 (socket says 172.21.15.19:38100) 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 -- 172.21.15.19:0/899429289 learned_addr learned my addr 172.21.15.19:0/899429289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa096ffd640 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa0977fe640 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 -- 172.21.15.19:0/899429289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 msgr2=0x56267df260b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df260b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 -- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 msgr2=0x56267df265f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df265f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 -- 172.21.15.19:0/899429289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267dcb7f70 con 0x56267db77410 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa097fff640 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2c230 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa09002fa50 tx=0x7fa09002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b765c9fe1d7ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.107+0000 7fa076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa076ffd640 1 -- 172.21.15.19:0/899429289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090060060 con 0x56267db77410 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa076ffd640 1 -- 172.21.15.19:0/899429289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa090031890 con 0x56267db77410 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09004e070 con 0x56267db77410 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa076ffd640 1 -- 172.21.15.19:0/899429289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa090003d40 con 0x56267db77410 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 10 client.16554 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56267db4f170 con 0x56267db77410 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa076ffd640 1 -- 172.21.15.19:0/899429289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09004e070 con 0x56267db77410 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa076ffd640 1 -- 172.21.15.19:0/899429289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa0900043f0 con 0x56267db77410 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 10 client.16554 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 20 client.16554 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db361f0 0x56267df38300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 4 client.16554 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.108+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56267de8b710 con 0x56267db361f0 2024-04-25T11:01:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.109+0000 7fa0977fe640 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db361f0 0x56267df38300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.109+0000 7fa0977fe640 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db361f0 0x56267df38300 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902378e0d11e736f server_cookie=e52b9b278f05571 in_seq=0 out_seq=0 2024-04-25T11:01:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.109+0000 7fa076ffd640 10 client.16554 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa076ffd640 1 -- 172.21.15.19:0/899429289 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56267de8b710 con 0x56267db361f0 2024-04-25T11:01:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa076ffd640 10 client.16554 handle_command_reply: tid=0 2024-04-25T11:01:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 client.16554 shutdown 2024-04-25T11:01:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 msgr2=0x56267df2c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2c230 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa09002fa50 tx=0x7fa09002f310 comp rx=0 tx=0).stop 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 shutdown_connections 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db361f0 0x56267df38300 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2c230 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db56730 0x56267df265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa076ffd640 0 client.16554 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 --2- 172.21.15.19:0/899429289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db4e550 0x56267df260b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 >> 172.21.15.19:0/899429289 conn(0x56267db6b560 msgr2=0x56267df19200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 shutdown_connections 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.110+0000 7fa0a5057740 1 -- 172.21.15.19:0/899429289 wait complete. 2024-04-25T11:01:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.111+0000 7fa0a5057740 20 client.16554 trim_cache size 0 max 16384 2024-04-25T11:01:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.129+0000 7fa0a5057740 1 Processor -- start 2024-04-25T11:01:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0a5057740 1 -- start start 2024-04-25T11:01:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df25650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db56730 0x56267df1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db77410 0x56267df1fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0a5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981ab640 con 0x56267db56730 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0a5057740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981adec0 con 0x56267db4e550 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0a5057740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981abcb0 con 0x56267db77410 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db56730 0x56267df1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db56730 0x56267df1f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38106/0 (socket says 172.21.15.19:38106) 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 -- 172.21.15.19:0/274921168 learned_addr learned my addr 172.21.15.19:0/274921168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa0977fe640 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df25650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa097fff640 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db77410 0x56267df1fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 -- 172.21.15.19:0/274921168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db77410 msgr2=0x56267df1fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db77410 0x56267df1fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 -- 172.21.15.19:0/274921168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 msgr2=0x56267df25650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df25650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 -- 172.21.15.19:0/274921168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267df14eb0 con 0x56267db56730 2024-04-25T11:01:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.130+0000 7fa096ffd640 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db56730 0x56267df1f860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0a6b90 tx=0x7fa08c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d439ae5ee0c536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.131+0000 7fa07d7fa640 1 -- 172.21.15.19:0/274921168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0e0040 con 0x56267db56730 2024-04-25T11:01:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.131+0000 7fa07d7fa640 1 -- 172.21.15.19:0/274921168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c009de0 con 0x56267db56730 2024-04-25T11:01:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.131+0000 7fa07d7fa640 1 -- 172.21.15.19:0/274921168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0dcdf0 con 0x56267db56730 2024-04-25T11:01:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.131+0000 7fa0a5057740 1 -- 172.21.15.19:0/274921168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db56730 msgr2=0x56267df1f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.131+0000 7fa0a5057740 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db56730 0x56267df1f860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0a6b90 tx=0x7fa08c00bf60 comp rx=0 tx=0).stop 2024-04-25T11:01:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- 172.21.15.19:0/274921168 shutdown_connections 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db77410 0x56267df1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db56730 0x56267df1f860 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 --2- 172.21.15.19:0/274921168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df25650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- 172.21.15.19:0/274921168 >> 172.21.15.19:0/274921168 conn(0x56267db6b560 msgr2=0x56267df17de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- 172.21.15.19:0/274921168 shutdown_connections 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- 172.21.15.19:0/274921168 wait complete. 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- start start 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 Processor -- start 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df2e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db56730 0x56267df2e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981ac380 con 0x56267db77410 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981ad7f0 con 0x56267db4e550 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.132+0000 7fa0a5057740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981aca50 con 0x56267db56730 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa0a5057740 10 client.0 _ll_register_callbacks cb 0x7fa090031d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38120/0 (socket says 172.21.15.19:38120) 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 -- 172.21.15.19:0/2606324065 learned_addr learned my addr 172.21.15.19:0/2606324065 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa096ffd640 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db56730 0x56267df2e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa0977fe640 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df2e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 -- 172.21.15.19:0/2606324065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db56730 msgr2=0x56267df2e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db56730 0x56267df2e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 -- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 msgr2=0x56267df2e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df2e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 -- 172.21.15.19:0/2606324065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267df9f620 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa097fff640 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2ebf0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa090006010 tx=0x7fa0900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec04af5f1f1c12b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa076ffd640 1 -- 172.21.15.19:0/2606324065 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090060070 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa08c009de0 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa076ffd640 1 -- 172.21.15.19:0/2606324065 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa090030e30 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa076ffd640 1 -- 172.21.15.19:0/2606324065 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa090031e10 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa0a5057740 10 client.16560 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.133+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56267dfa08a0 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa076ffd640 1 -- 172.21.15.19:0/2606324065 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c009de0 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa076ffd640 1 -- 172.21.15.19:0/2606324065 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa0900040d0 con 0x56267db77410 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0a5057740 10 client.16560 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0a5057740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0a5057740 20 client.16560 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0a5057740 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db0b630 0x56267dfa2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0a5057740 4 client.16560 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56267de85580 con 0x56267db0b630 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0977fe640 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db0b630 0x56267dfa2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa076ffd640 10 client.16560 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.134+0000 7fa0977fe640 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db0b630 0x56267dfa2f40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39381f704284f80 server_cookie=e2ab45972a221d4a in_seq=0 out_seq=0 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa076ffd640 1 -- 172.21.15.19:0/2606324065 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56267de85580 con 0x56267db0b630 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa076ffd640 10 client.16560 handle_command_reply: tid=0 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 client.16560 shutdown 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 msgr2=0x56267df2ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2ebf0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa090006010 tx=0x7fa0900047c0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 shutdown_connections 2024-04-25T11:01:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56267db0b630 0x56267dfa2f40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56267db77410 0x56267df2ebf0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56267db56730 0x56267df2e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 --2- 172.21.15.19:0/2606324065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56267db4e550 0x56267df2e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 >> 172.21.15.19:0/2606324065 conn(0x56267db6b560 msgr2=0x56267df190f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa076ffd640 0 client.16560 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 shutdown_connections 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 1 -- 172.21.15.19:0/2606324065 wait complete. 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.136+0000 7fa0a5057740 20 client.16560 trim_cache size 0 max 16384 2024-04-25T11:01:01.143 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:01:01.144 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:01:01.144 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "11b20a2e-ca31-48e8-8812-79d14825aad8", 2024-04-25T11:01:01.144 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:01:01.144 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa0800afc60 msgr2=0x7fa0800b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa0800afc60 0x7fa0800b2170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa09004d000 tx=0x7fa09004b000 comp rx=0 tx=0).stop 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098109b20 msgr2=0x7fa09819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098109b20 0x7fa09819ac50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00d950 tx=0x7fa08c00de10 comp rx=0 tx=0).stop 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 shutdown_connections 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa0800afc60 0x7fa0800b2170 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa098109fc0 0x7fa09819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa098109b20 0x7fa09819ac50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 --2- 172.21.15.19:0/3900146979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa098104270 0x7fa09819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 >> 172.21.15.19:0/3900146979 conn(0x7fa0980ff1e0 msgr2=0x7fa098106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 shutdown_connections 2024-04-25T11:01:01.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.137+0000 7fa09e1a8640 1 -- 172.21.15.19:0/3900146979 wait complete. 2024-04-25T11:01:01.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:01:01.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 -- 172.21.15.19:0/986788032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f0108a20 msgr2=0x7f27f01025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 --2- 172.21.15.19:0/986788032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f0108a20 0x7f27f01025b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e002a410 comp rx=0 tx=0).stop 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 -- 172.21.15.19:0/986788032 shutdown_connections 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 --2- 172.21.15.19:0/986788032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27f010d380 0x7f27f010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 --2- 172.21.15.19:0/986788032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f0102af0 0x7f27f0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 --2- 172.21.15.19:0/986788032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f0108a20 0x7f27f01025b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 -- 172.21.15.19:0/986788032 >> 172.21.15.19:0/986788032 conn(0x7f27f0078100 msgr2=0x7f27f0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 -- 172.21.15.19:0/986788032 shutdown_connections 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.262+0000 7f27f5220640 1 -- 172.21.15.19:0/986788032 wait complete. 2024-04-25T11:01:01.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 Processor -- start 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 -- start start 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27f0102af0 0x7f27f019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f0108a20 0x7f27f019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f010d380 0x7f27f019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0112be0 con 0x7f27f010d380 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0112a60 con 0x7f27f0102af0 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27f5220640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0112d60 con 0x7f27f0108a20 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27ef59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f010d380 0x7f27f019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27ef59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f010d380 0x7f27f019c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38138/0 (socket says 172.21.15.19:38138) 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.263+0000 7f27ef59e640 1 -- 172.21.15.19:0/2746821098 learned_addr learned my addr 172.21.15.19:0/2746821098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27eed9d640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27f0102af0 0x7f27f019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27ee59c640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f0108a20 0x7f27f019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27ef59e640 1 -- 172.21.15.19:0/2746821098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f0108a20 msgr2=0x7f27f019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27ef59e640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f0108a20 0x7f27f019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27ef59e640 1 -- 172.21.15.19:0/2746821098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27f0102af0 msgr2=0x7f27f019b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27ef59e640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27f0102af0 0x7f27f019b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27ef59e640 1 -- 172.21.15.19:0/2746821098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f01a6880 con 0x7f27f010d380 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27ef59e640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f010d380 0x7f27f019c040 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f27e800eaa0 tx=0x7f27e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27d7fff640 1 -- 172.21.15.19:0/2746821098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e800cdb0 con 0x7f27f010d380 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27d7fff640 1 -- 172.21.15.19:0/2746821098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e8005020 con 0x7f27f010d380 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f01a6b70 con 0x7f27f010d380 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27d7fff640 1 -- 172.21.15.19:0/2746821098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80116d0 con 0x7f27f010d380 2024-04-25T11:01:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.264+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f01a7070 con 0x7f27f010d380 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 Processor -- start 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27d7fff640 1 -- 172.21.15.19:0/2746821098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f27e8011870 con 0x7f27f010d380 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 -- start start 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56554d90ed80 con 0x56554d5e76b0 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56554d8c88b0 con 0x56554d5df4d0 2024-04-25T11:01:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.267+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56554d90ad40 con 0x56554d608390 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38140/0 (socket says 172.21.15.19:38140) 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27ee59c640 1 -- 172.21.15.19:0/2668561086 learned_addr learned my addr 172.21.15.19:0/2668561086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27ee59c640 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27d7fff640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f27d80afc60 0x7f27d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27eed9d640 1 -- 172.21.15.19:0/2668561086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 msgr2=0x56554d9a8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27eed9d640 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9a8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27eed9d640 1 -- 172.21.15.19:0/2668561086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 msgr2=0x56554d9b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27eed9d640 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27eed9d640 1 -- 172.21.15.19:0/2668561086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554d899a40 con 0x56554d5df4d0 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27ee59c640 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27ef59e640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f27d80afc60 0x7f27d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27d7fff640 1 -- 172.21.15.19:0/2746821098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f27e80df1e0 con 0x7f27f010d380 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.268+0000 7f27eed9d640 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9a8fa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27e0006010 tx=0x7f27e0005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0c3d085cc8c54c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27d57fa640 1 -- 172.21.15.19:0/2668561086 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0040070 con 0x56554d5df4d0 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27ef59e640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f27d80afc60 0x7f27d80b2170 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f27e80a33d0 tx=0x7f27e8074bb0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27d57fa640 1 -- 172.21.15.19:0/2668561086 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e0004030 con 0x56554d5df4d0 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27d57fa640 1 -- 172.21.15.19:0/2668561086 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0035040 con 0x56554d5df4d0 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27fc1ac740 1 -- 172.21.15.19:0/2668561086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 msgr2=0x56554d9a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9a8fa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27e0006010 tx=0x7f27e0005ef0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27fc1ac740 1 -- 172.21.15.19:0/2668561086 shutdown_connections 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9a8fa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/2668561086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9a8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.269+0000 7f27fc1ac740 1 -- 172.21.15.19:0/2668561086 >> 172.21.15.19:0/2668561086 conn(0x56554d5fc4e0 msgr2=0x56554d9a5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 -- 172.21.15.19:0/2668561086 shutdown_connections 2024-04-25T11:01:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 -- 172.21.15.19:0/2668561086 wait complete. 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 -- start start 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 Processor -- start 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d5df4d0 0x56554d9b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554d9b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d608390 0x56554d9b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56554d9b5f80 con 0x56554d608390 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56554d895d10 con 0x56554d5e76b0 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56554d9b6100 con 0x56554d5df4d0 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27fc1ac740 10 client.0 _ll_register_callbacks cb 0x56554d8c2210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d608390 0x56554d9b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d608390 0x56554d9b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38146/0 (socket says 172.21.15.19:38146) 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27ee59c640 1 -- 172.21.15.19:0/4021161483 learned_addr learned my addr 172.21.15.19:0/4021161483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.270+0000 7f27ee59c640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d5df4d0 0x56554d9b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27eed9d640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554d9b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27ee59c640 1 -- 172.21.15.19:0/4021161483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d5df4d0 msgr2=0x56554d9b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27ee59c640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d5df4d0 0x56554d9b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27ee59c640 1 -- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 msgr2=0x56554d9b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27ee59c640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554d9b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27ee59c640 1 -- 172.21.15.19:0/4021161483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554d6ea7d0 con 0x56554d608390 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27eed9d640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554d9b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27ee59c640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d608390 0x56554d9b93d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f27e400ce80 tx=0x7f27e400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6246df8aad89e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27d2ff5640 1 -- 172.21.15.19:0/4021161483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40142d0 con 0x56554d608390 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27d2ff5640 1 -- 172.21.15.19:0/4021161483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f27e400f630 con 0x56554d608390 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e0004030 con 0x56554d608390 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27d2ff5640 1 -- 172.21.15.19:0/4021161483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f27e400f830 con 0x56554d608390 2024-04-25T11:01:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27fc1ac740 10 client.16572 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56554d6203f0 con 0x56554d608390 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.271+0000 7f27d2ff5640 1 -- 172.21.15.19:0/4021161483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e0004030 con 0x56554d608390 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27d2ff5640 1 -- 172.21.15.19:0/4021161483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f27e40127c0 con 0x56554d608390 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27fc1ac740 10 client.16572 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27fc1ac740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27fc1ac740 20 client.16572 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d5c6f60 0x56554d9c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27fc1ac740 4 client.16572 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56554d91c990 con 0x56554d5c6f60 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27eed9d640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d5c6f60 0x56554d9c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27d2ff5640 10 client.16572 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.272+0000 7f27eed9d640 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d5c6f60 0x56554d9c56a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74075cf34eb33826 server_cookie=14185ed24791769d in_seq=0 out_seq=0 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.273+0000 7f27d2ff5640 1 -- 172.21.15.19:0/4021161483 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56554d91c990 con 0x56554d5c6f60 2024-04-25T11:01:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.273+0000 7f27d2ff5640 10 client.16572 handle_command_reply: tid=0 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.273+0000 7f27fc1ac740 1 client.16572 shutdown 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.273+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d608390 msgr2=0x56554d9b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.273+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d608390 0x56554d9b93d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f27e400ce80 tx=0x7f27e400cbb0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 shutdown_connections 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d5c6f60 0x56554d9c56a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d608390 0x56554d9b93d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554d9b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/4021161483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d5df4d0 0x56554d9b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27d2ff5640 0 client.16572 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 >> 172.21.15.19:0/4021161483 conn(0x56554d5fc4e0 msgr2=0x56554d9aa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 shutdown_connections 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 1 -- 172.21.15.19:0/4021161483 wait complete. 2024-04-25T11:01:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.274+0000 7f27fc1ac740 20 client.16572 trim_cache size 0 max 16384 2024-04-25T11:01:01.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 Processor -- start 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 -- start start 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9aad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01ad7c0 con 0x56554d5e76b0 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01b0040 con 0x56554d5df4d0 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01ade30 con 0x56554d608390 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.292+0000 7f27eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38150/0 (socket says 172.21.15.19:38150) 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 -- 172.21.15.19:0/1393317646 learned_addr learned my addr 172.21.15.19:0/1393317646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27ee59c640 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27ee59c640 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 -- 172.21.15.19:0/1393317646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 msgr2=0x56554d9cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 -- 172.21.15.19:0/1393317646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 msgr2=0x56554d9b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 -- 172.21.15.19:0/1393317646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554d5e00f0 con 0x56554d5e76b0 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27eed9d640 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9aad70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009400 tx=0x7f27e0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a15ef2d43bccea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27d4ff9640 1 -- 172.21.15.19:0/1393317646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0035040 con 0x56554d5e76b0 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27d4ff9640 1 -- 172.21.15.19:0/1393317646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e0052070 con 0x56554d5e76b0 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.293+0000 7f27d4ff9640 1 -- 172.21.15.19:0/1393317646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0040040 con 0x56554d5e76b0 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 -- 172.21.15.19:0/1393317646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 msgr2=0x56554d9aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9aad70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009400 tx=0x7f27e0031a50 comp rx=0 tx=0).stop 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 -- 172.21.15.19:0/1393317646 shutdown_connections 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5e76b0 0x56554d9aad70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/1393317646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5df4d0 0x56554d9b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 -- 172.21.15.19:0/1393317646 >> 172.21.15.19:0/1393317646 conn(0x56554d5fc4e0 msgr2=0x56554d9aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 -- 172.21.15.19:0/1393317646 shutdown_connections 2024-04-25T11:01:01.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 -- 172.21.15.19:0/1393317646 wait complete. 2024-04-25T11:01:01.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 -- start start 2024-04-25T11:01:01.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.294+0000 7f27fc1ac740 1 Processor -- start 2024-04-25T11:01:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5df4d0 0x56554da2a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554da2a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27fc1ac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9bf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01ae500 con 0x56554d5df4d0 2024-04-25T11:01:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01af970 con 0x56554d5e76b0 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27fc1ac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01aebd0 con 0x56554d608390 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27fc1ac740 10 client.0 _ll_register_callbacks cb 0x7f27e400cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5df4d0 0x56554da2a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5df4d0 0x56554da2a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38162/0 (socket says 172.21.15.19:38162) 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 -- 172.21.15.19:0/367375754 learned_addr learned my addr 172.21.15.19:0/367375754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27eed9d640 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554da2a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 -- 172.21.15.19:0/367375754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 msgr2=0x56554d9bf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9bf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 -- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 msgr2=0x56554da2a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554da2a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 -- 172.21.15.19:0/367375754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554d8cf4b0 con 0x56554d5df4d0 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27ee59c640 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5df4d0 0x56554da2a320 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f27e4012dd0 tx=0x7f27e4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee813ffc951e7b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.295+0000 7f27d2ff5640 1 -- 172.21.15.19:0/367375754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e403b060 con 0x56554d5df4d0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27d2ff5640 1 -- 172.21.15.19:0/367375754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f27e40140d0 con 0x56554d5df4d0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e0052070 con 0x56554d5df4d0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27d2ff5640 1 -- 172.21.15.19:0/367375754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f27e402e050 con 0x56554d5df4d0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 10 client.16578 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56554da31790 con 0x56554d5df4d0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27d2ff5640 1 -- 172.21.15.19:0/367375754 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e0052070 con 0x56554d5df4d0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27d2ff5640 1 -- 172.21.15.19:0/367375754 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f27e400fc70 con 0x56554d5df4d0 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 10 client.16578 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 20 client.16578 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d59c3a0 0x56554da33e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 4 client.16578 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56554d916800 con 0x56554d59c3a0 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.296+0000 7f27eed9d640 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d59c3a0 0x56554da33e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27eed9d640 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d59c3a0 0x56554da33e00 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537782c6c878febd server_cookie=e06241eca426cc29 in_seq=0 out_seq=0 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27d2ff5640 10 client.16578 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27d2ff5640 1 -- 172.21.15.19:0/367375754 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56554d916800 con 0x56554d59c3a0 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27d2ff5640 10 client.16578 handle_command_reply: tid=0 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 client.16578 shutdown 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5df4d0 msgr2=0x56554da2a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5df4d0 0x56554da2a320 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f27e4012dd0 tx=0x7f27e4011ec0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 shutdown_connections 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56554d59c3a0 0x56554da33e00 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56554d608390 0x56554d9bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56554d5e76b0 0x56554da2a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27d2ff5640 0 client.16578 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 --2- 172.21.15.19:0/367375754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56554d5df4d0 0x56554da2a320 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 >> 172.21.15.19:0/367375754 conn(0x56554d5fc4e0 msgr2=0x56554d9b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 shutdown_connections 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.297+0000 7f27fc1ac740 1 -- 172.21.15.19:0/367375754 wait complete. 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27fc1ac740 20 client.16578 trim_cache size 0 max 16384 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "11b20a2e-ca31-48e8-8812-79d14825aad8": { 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:01:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "11b20a2e-ca31-48e8-8812-79d14825aad8", 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f27d80afc60 msgr2=0x7f27d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f27d80afc60 0x7f27d80b2170 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f27e80a33d0 tx=0x7f27e8074bb0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f010d380 msgr2=0x7f27f019c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f010d380 0x7f27f019c040 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f27e800eaa0 tx=0x7f27e800ef60 comp rx=0 tx=0).stop 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 shutdown_connections 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f27d80afc60 0x7f27d80b2170 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f010d380 0x7f27f019c040 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f0108a20 0x7f27f019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 --2- 172.21.15.19:0/2746821098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27f0102af0 0x7f27f019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 >> 172.21.15.19:0/2746821098 conn(0x7f27f0078100 msgr2=0x7f27f01040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 shutdown_connections 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:01.298+0000 7f27f5220640 1 -- 172.21.15.19:0/2746821098 wait complete. 2024-04-25T11:01:01.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:01 smithi019 ceph-mon[29209]: pgmap v198: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:01.310 INFO:tasks.cephfs.filesystem:scrub status for tag:11b20a2e-ca31-48e8-8812-79d14825aad8 - {'path': '/', 'tag': '11b20a2e-ca31-48e8-8812-79d14825aad8', 'options': 'recursive,force'} 2024-04-25T11:01:01.367 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:01 smithi169 ceph-mon[33049]: pgmap v198: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:01.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:01 smithi112 ceph-mon[32799]: pgmap v198: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:02.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:02 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:01:02.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:02 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:02.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:02 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:01:02.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:02 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:02.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:02 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:01:02.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:02 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:01:02.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:02 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:02.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:02 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:01:02.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:02 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:02.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:02 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:01:02.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:02 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:01:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:02 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:02 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:01:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:02 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:02.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:02 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:01:03.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:03 smithi019 ceph-mon[29209]: pgmap v199: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:01:03.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:03 smithi169 ceph-mon[33049]: pgmap v199: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:01:03.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:03 smithi112 ceph-mon[32799]: pgmap v199: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:01:05.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:05 smithi169 ceph-mon[33049]: pgmap v200: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T11:01:05.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:05 smithi112 ceph-mon[32799]: pgmap v200: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T11:01:05.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:05 smithi019 ceph-mon[29209]: pgmap v200: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T11:01:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:07 smithi169 ceph-mon[33049]: pgmap v201: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-04-25T11:01:07.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:07 smithi112 ceph-mon[32799]: pgmap v201: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-04-25T11:01:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:07 smithi019 ceph-mon[29209]: pgmap v201: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-04-25T11:01:08.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:08 smithi169 ceph-mon[33049]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:01:08.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:08 smithi112 ceph-mon[32799]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:01:08.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:08 smithi019 ceph-mon[29209]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:01:09.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:09 smithi169 ceph-mon[33049]: pgmap v203: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:01:09.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:09 smithi112 ceph-mon[32799]: pgmap v203: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:01:09.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:09 smithi019 ceph-mon[29209]: pgmap v203: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:01:11.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:11 smithi169 ceph-mon[33049]: pgmap v204: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:01:11.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:11 smithi112 ceph-mon[32799]: pgmap v204: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:01:11.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:11 smithi019 ceph-mon[29209]: pgmap v204: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:01:13.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:13 smithi169 ceph-mon[33049]: pgmap v205: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:01:13.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:13 smithi169 ceph-mon[33049]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:01:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:13 smithi112 ceph-mon[32799]: pgmap v205: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:01:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:13 smithi112 ceph-mon[32799]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:01:13.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:13 smithi019 ceph-mon[29209]: pgmap v205: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:01:13.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:13 smithi019 ceph-mon[29209]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:01:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:15 smithi169 ceph-mon[33049]: pgmap v207: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:01:15.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:15 smithi112 ceph-mon[32799]: pgmap v207: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:01:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:15 smithi019 ceph-mon[29209]: pgmap v207: 97 pgs: 97 active+clean; 11 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:01:17.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:17 smithi169 ceph-mon[33049]: pgmap v208: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 229 B/s wr, 2 op/s 2024-04-25T11:01:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:17 smithi112 ceph-mon[32799]: pgmap v208: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 229 B/s wr, 2 op/s 2024-04-25T11:01:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:17 smithi019 ceph-mon[29209]: pgmap v208: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 229 B/s wr, 2 op/s 2024-04-25T11:01:19.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:19 smithi169 ceph-mon[33049]: pgmap v209: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:01:19.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:19 smithi112 ceph-mon[32799]: pgmap v209: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:01:19.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:19 smithi019 ceph-mon[29209]: pgmap v209: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:01:21.413 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:21 smithi019 ceph-mon[29209]: pgmap v210: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:01:21.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:21 smithi169 ceph-mon[33049]: pgmap v210: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:01:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:21 smithi112 ceph-mon[32799]: pgmap v210: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:01:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:23 smithi169 ceph-mon[33049]: pgmap v211: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:01:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:23 smithi112 ceph-mon[32799]: pgmap v211: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:01:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:23 smithi019 ceph-mon[29209]: pgmap v211: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:01:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:25 smithi169 ceph-mon[33049]: pgmap v212: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:01:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:25 smithi112 ceph-mon[32799]: pgmap v212: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:01:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:25 smithi019 ceph-mon[29209]: pgmap v212: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:01:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:27 smithi169 ceph-mon[33049]: pgmap v213: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:01:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:27 smithi112 ceph-mon[32799]: pgmap v213: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:01:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:27 smithi019 ceph-mon[29209]: pgmap v213: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:01:29.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:29 smithi169 ceph-mon[33049]: pgmap v214: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:29.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:29 smithi112 ceph-mon[32799]: pgmap v214: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:29.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:29 smithi019 ceph-mon[29209]: pgmap v214: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:31.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:01:31.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:01:31.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.443+0000 7f875e60d640 1 -- 172.21.15.19:0/460556341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 msgr2=0x7f87581025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.443+0000 7f875e60d640 1 --2- 172.21.15.19:0/460556341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f87581025b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f874802a410 comp rx=0 tx=0).stop 2024-04-25T11:01:31.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 -- 172.21.15.19:0/460556341 shutdown_connections 2024-04-25T11:01:31.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 --2- 172.21.15.19:0/460556341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f875810d380 0x7f875810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 --2- 172.21.15.19:0/460556341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8758102af0 0x7f8758102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 --2- 172.21.15.19:0/460556341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f87581025b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 -- 172.21.15.19:0/460556341 >> 172.21.15.19:0/460556341 conn(0x7f8758078100 msgr2=0x7f8758078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 -- 172.21.15.19:0/460556341 shutdown_connections 2024-04-25T11:01:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 -- 172.21.15.19:0/460556341 wait complete. 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 Processor -- start 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.444+0000 7f875e60d640 1 -- start start 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f875e60d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8758102af0 0x7f875819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f875e60d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f875819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f875e60d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875810d380 0x7f875819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f875e60d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8758112c30 con 0x7f8758108a20 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f875e60d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8758112ab0 con 0x7f875810d380 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f875e60d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8758112db0 con 0x7f8758102af0 2024-04-25T11:01:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f875819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f875819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41456/0 (socket says 172.21.15.19:41456) 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 -- 172.21.15.19:0/1829218160 learned_addr learned my addr 172.21.15.19:0/1829218160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f875cbaa640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875810d380 0x7f875819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f8757fff640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8758102af0 0x7f875819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 -- 172.21.15.19:0/1829218160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8758102af0 msgr2=0x7f875819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8758102af0 0x7f875819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 -- 172.21.15.19:0/1829218160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875810d380 msgr2=0x7f875819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875810d380 0x7f875819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.445+0000 7f87577fe640 1 -- 172.21.15.19:0/1829218160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875819f560 con 0x7f8758108a20 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.446+0000 7f87577fe640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f875819ac60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c970 tx=0x7f874c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.446+0000 7f87557fa640 1 -- 172.21.15.19:0/1829218160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c004220 con 0x7f8758108a20 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.446+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875819f850 con 0x7f8758108a20 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.446+0000 7f87557fa640 1 -- 172.21.15.19:0/1829218160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f874c0043c0 con 0x7f8758108a20 2024-04-25T11:01:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.446+0000 7f87557fa640 1 -- 172.21.15.19:0/1829218160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c00f450 con 0x7f8758108a20 2024-04-25T11:01:31.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.446+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581a71d0 con 0x7f8758108a20 2024-04-25T11:01:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.448+0000 7f8765595740 1 Processor -- start 2024-04-25T11:01:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8765595740 1 -- start start 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f87557fa640 1 -- 172.21.15.19:0/1829218160 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f874c00f5f0 con 0x7f8758108a20 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8765595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52b5390 0x55dfd5655a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8765595740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd528c4d0 0x55dfd5655fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8765595740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd56628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8765595740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dfd55bbd80 con 0x55dfd528c4d0 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8765595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dfd55758b0 con 0x55dfd52b5390 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8765595740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dfd55b7d40 con 0x55dfd52946b0 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8757fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd528c4d0 0x55dfd5655fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f875cbaa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52b5390 0x55dfd5655a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8757fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd528c4d0 0x55dfd5655fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41466/0 (socket says 172.21.15.19:41466) 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8757fff640 1 -- 172.21.15.19:0/3550195360 learned_addr learned my addr 172.21.15.19:0/3550195360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f87577fe640 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd56628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f875cbaa640 1 -- 172.21.15.19:0/3550195360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 msgr2=0x55dfd56628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f875cbaa640 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd56628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f875cbaa640 1 -- 172.21.15.19:0/3550195360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd528c4d0 msgr2=0x55dfd5655fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f875cbaa640 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd528c4d0 0x55dfd5655fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f875cbaa640 1 -- 172.21.15.19:0/3550195360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd5546a40 con 0x55dfd52b5390 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f8757fff640 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd528c4d0 0x55dfd5655fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.449+0000 7f87557fa640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87400afc60 0x7f87400b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.450+0000 7f875cbaa640 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52b5390 0x55dfd5655a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8750009f30 tx=0x7f8750010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b2a5a30e66a51a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.450+0000 7f8757fff640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87400afc60 0x7f87400b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.450+0000 7f87557fa640 1 -- 172.21.15.19:0/1829218160 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f874c0df0c0 con 0x7f8758108a20 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.450+0000 7f873e7fc640 1 -- 172.21.15.19:0/3550195360 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750016070 con 0x55dfd52b5390 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.450+0000 7f873e7fc640 1 -- 172.21.15.19:0/3550195360 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8750004d20 con 0x55dfd52b5390 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.450+0000 7f873e7fc640 1 -- 172.21.15.19:0/3550195360 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750012740 con 0x55dfd52b5390 2024-04-25T11:01:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 -- 172.21.15.19:0/3550195360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52b5390 msgr2=0x55dfd5655a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8757fff640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87400afc60 0x7f87400b2170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55dfd56531a0 tx=0x7f8748009640 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52b5390 0x55dfd5655a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8750009f30 tx=0x7f8750010800 comp rx=0 tx=0).stop 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 -- 172.21.15.19:0/3550195360 shutdown_connections 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd56628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd528c4d0 0x55dfd5655fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 --2- 172.21.15.19:0/3550195360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52b5390 0x55dfd5655a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 -- 172.21.15.19:0/3550195360 >> 172.21.15.19:0/3550195360 conn(0x55dfd52a94e0 msgr2=0x55dfd5652210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 -- 172.21.15.19:0/3550195360 shutdown_connections 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 -- 172.21.15.19:0/3550195360 wait complete. 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 -- start start 2024-04-25T11:01:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.451+0000 7f8765595740 1 Processor -- start 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8765595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8765595740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd5662540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8765595740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8765595740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dfd5662fe0 con 0x55dfd52b5390 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8765595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dfd5542d10 con 0x55dfd528c4d0 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8765595740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dfd5663160 con 0x55dfd52946b0 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8765595740 10 client.0 _ll_register_callbacks cb 0x55dfd556f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41474/0 (socket says 172.21.15.19:41474) 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 -- 172.21.15.19:0/1676858557 learned_addr learned my addr 172.21.15.19:0/1676858557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f8757fff640 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f87577fe640 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd5662540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 -- 172.21.15.19:0/1676858557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 msgr2=0x55dfd5662540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd5662540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 -- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 msgr2=0x55dfd5662000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 -- 172.21.15.19:0/1676858557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd53977d0 con 0x55dfd52b5390 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f875cbaa640 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56604f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8750000f80 tx=0x7f8750005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dc0986aa7ea45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.452+0000 7f87377fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f87377fe640 1 -- 172.21.15.19:0/1676858557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750029060 con 0x55dfd52b5390 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f87377fe640 1 -- 172.21.15.19:0/1676858557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f8750005480 con 0x55dfd52b5390 2024-04-25T11:01:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8750004d20 con 0x55dfd52b5390 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f87377fe640 1 -- 172.21.15.19:0/1676858557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8750016070 con 0x55dfd52b5390 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 10 client.16590 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dfd52cd3f0 con 0x55dfd52b5390 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f87377fe640 1 -- 172.21.15.19:0/1676858557 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8750004d20 con 0x55dfd52b5390 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f87377fe640 1 -- 172.21.15.19:0/1676858557 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f875000f040 con 0x55dfd52b5390 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 10 client.16590 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 20 client.16590 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd5273f60 0x55dfd56726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 4 client.16590 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.453+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfd55c9990 con 0x55dfd5273f60 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.454+0000 7f8757fff640 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd5273f60 0x55dfd56726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.454+0000 7f87377fe640 10 client.16590 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.454+0000 7f8757fff640 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd5273f60 0x55dfd56726b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673996b8cec7acf5 server_cookie=85222b0ef7eb7371 in_seq=0 out_seq=0 2024-04-25T11:01:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f87377fe640 1 -- 172.21.15.19:0/1676858557 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dfd55c9990 con 0x55dfd5273f60 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f87377fe640 10 client.16590 handle_command_reply: tid=0 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 client.16590 shutdown 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 msgr2=0x55dfd56604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56604f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8750000f80 tx=0x7f8750005100 comp rx=0 tx=0).stop 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 shutdown_connections 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd5273f60 0x55dfd56726b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56604f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f87377fe640 0 client.16590 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd5662540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 --2- 172.21.15.19:0/1676858557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 >> 172.21.15.19:0/1676858557 conn(0x55dfd52a94e0 msgr2=0x55dfd5657780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 shutdown_connections 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 1 -- 172.21.15.19:0/1676858557 wait complete. 2024-04-25T11:01:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.455+0000 7f8765595740 20 client.16590 trim_cache size 0 max 16384 2024-04-25T11:01:31.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 Processor -- start 2024-04-25T11:01:31.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 -- start start 2024-04-25T11:01:31.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd5657d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87581ad940 con 0x55dfd52b5390 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87581b01a0 con 0x55dfd528c4d0 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8765595740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f87581adf90 con 0x55dfd52946b0 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56756e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41490/0 (socket says 172.21.15.19:41490) 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 -- 172.21.15.19:0/3658993357 learned_addr learned my addr 172.21.15.19:0/3658993357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f8757fff640 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 -- 172.21.15.19:0/3658993357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 msgr2=0x55dfd5657d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd5657d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 -- 172.21.15.19:0/3658993357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 msgr2=0x55dfd5662670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.474+0000 7f875cbaa640 1 -- 172.21.15.19:0/3658993357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd528d0f0 con 0x55dfd52b5390 2024-04-25T11:01:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.475+0000 7f875cbaa640 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56756e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8750009640 tx=0x7f8750010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4129676405e15858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.475+0000 7f873dffb640 1 -- 172.21.15.19:0/3658993357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750029040 con 0x55dfd52b5390 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.475+0000 7f873dffb640 1 -- 172.21.15.19:0/3658993357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875001ae50 con 0x55dfd52b5390 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.475+0000 7f873dffb640 1 -- 172.21.15.19:0/3658993357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750005480 con 0x55dfd52b5390 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.475+0000 7f8765595740 1 -- 172.21.15.19:0/3658993357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 msgr2=0x55dfd56756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.475+0000 7f8765595740 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56756e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8750009640 tx=0x7f8750010800 comp rx=0 tx=0).stop 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.475+0000 7f8765595740 1 -- 172.21.15.19:0/3658993357 shutdown_connections 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56756e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd52946b0 0x55dfd5657d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 --2- 172.21.15.19:0/3658993357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd528c4d0 0x55dfd5662670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 -- 172.21.15.19:0/3658993357 >> 172.21.15.19:0/3658993357 conn(0x55dfd52a94e0 msgr2=0x55dfd5657660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 -- 172.21.15.19:0/3658993357 shutdown_connections 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 -- 172.21.15.19:0/3658993357 wait complete. 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 -- start start 2024-04-25T11:01:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.476+0000 7f8765595740 1 Processor -- start 2024-04-25T11:01:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd528c4d0 0x55dfd56749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52946b0 0x55dfd5674f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87581ae660 con 0x55dfd52b5390 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87581afad0 con 0x55dfd52946b0 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f87581aed30 con 0x55dfd528c4d0 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 10 client.0 _ll_register_callbacks cb 0x7f8748009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56d87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41500/0 (socket says 172.21.15.19:41500) 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 -- 172.21.15.19:0/1963359673 learned_addr learned my addr 172.21.15.19:0/1963359673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f87577fe640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52946b0 0x55dfd5674f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8757fff640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd528c4d0 0x55dfd56749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 -- 172.21.15.19:0/1963359673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd528c4d0 msgr2=0x55dfd56749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd528c4d0 0x55dfd56749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 -- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52946b0 msgr2=0x55dfd5674f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52946b0 0x55dfd5674f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 -- 172.21.15.19:0/1963359673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd557c4b0 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f87577fe640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52946b0 0x55dfd5674f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f875cbaa640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56d87e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8750005140 tx=0x7f8750004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3197d1d308705a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f87377fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f87377fe640 1 -- 172.21.15.19:0/1963359673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875001a720 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f87377fe640 1 -- 172.21.15.19:0/1963359673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f8750021070 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.477+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875001ae50 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f87377fe640 1 -- 172.21.15.19:0/1963359673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8750003b80 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 10 client.16596 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dfd56d9d60 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f87377fe640 1 -- 172.21.15.19:0/1963359673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875001ae50 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f87377fe640 1 -- 172.21.15.19:0/1963359673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8750012720 con 0x55dfd52b5390 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 10 client.16596 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:31.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 20 client.16596 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd52493a0 0x55dfd56dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 4 client.16596 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dfd55c3800 con 0x55dfd52493a0 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.478+0000 7f8757fff640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd52493a0 0x55dfd56dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f87377fe640 10 client.16596 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8757fff640 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd52493a0 0x55dfd56dc420 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c7e811b98aa01f server_cookie=6db91b9af0a40499 in_seq=0 out_seq=0 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f87377fe640 1 -- 172.21.15.19:0/1963359673 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dfd55c3800 con 0x55dfd52493a0 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f87377fe640 10 client.16596 handle_command_reply: tid=0 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 client.16596 shutdown 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 msgr2=0x55dfd56d87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56d87e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8750005140 tx=0x7f8750004e50 comp rx=0 tx=0).stop 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 shutdown_connections 2024-04-25T11:01:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55dfd52493a0 0x55dfd56dc420 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfd52b5390 0x55dfd56d87e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfd52946b0 0x55dfd5674f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f87377fe640 0 client.16596 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 --2- 172.21.15.19:0/1963359673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dfd528c4d0 0x55dfd56749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 >> 172.21.15.19:0/1963359673 conn(0x55dfd52a94e0 msgr2=0x55dfd565ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 shutdown_connections 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 1 -- 172.21.15.19:0/1963359673 wait complete. 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.479+0000 7f8765595740 20 client.16596 trim_cache size 0 max 16384 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:01:31.486 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:01:31.487 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:01:31.487 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:01:31.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87400afc60 msgr2=0x7f87400b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87400afc60 0x7f87400b2170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55dfd56531a0 tx=0x7f8748009640 comp rx=0 tx=0).stop 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 msgr2=0x7f875819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f875819ac60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c970 tx=0x7f874c00ce30 comp rx=0 tx=0).stop 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 shutdown_connections 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f87400afc60 0x7f87400b2170 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875810d380 0x7f875819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758108a20 0x7f875819ac60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 --2- 172.21.15.19:0/1829218160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8758102af0 0x7f875819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 >> 172.21.15.19:0/1829218160 conn(0x7f8758078100 msgr2=0x7f8758069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 shutdown_connections 2024-04-25T11:01:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.480+0000 7f875e60d640 1 -- 172.21.15.19:0/1829218160 wait complete. 2024-04-25T11:01:31.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:31 smithi019 ceph-mon[29209]: pgmap v215: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:31.492 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:01:31.492 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:01:31.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:31 smithi169 ceph-mon[33049]: pgmap v215: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:31.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:31 smithi112 ceph-mon[32799]: pgmap v215: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:31.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 -- 172.21.15.19:0/373034403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc10d3a0 msgr2=0x7fa3dc10f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/373034403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc10d3a0 0x7fa3dc10f7c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009ca0 tx=0x7fa3d402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 -- 172.21.15.19:0/373034403 shutdown_connections 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/373034403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc10d3a0 0x7fa3dc10f7c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/373034403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3dc102b60 0x7fa3dc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/373034403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3dc108a40 0x7fa3dc102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 -- 172.21.15.19:0/373034403 >> 172.21.15.19:0/373034403 conn(0x7fa3dc078040 msgr2=0x7fa3dc078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 -- 172.21.15.19:0/373034403 shutdown_connections 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.604+0000 7fa3e15ee640 1 -- 172.21.15.19:0/373034403 wait complete. 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 Processor -- start 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 -- start start 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3dc102b60 0x7fa3dc199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc108a40 0x7fa3dc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3dc10d3a0 0x7fa3dc19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc112c20 con 0x7fa3dc108a40 2024-04-25T11:01:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc112aa0 con 0x7fa3dc10d3a0 2024-04-25T11:01:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3e15ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc112da0 con 0x7fa3dc102b60 2024-04-25T11:01:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc108a40 0x7fa3dc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc108a40 0x7fa3dc199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41510/0 (socket says 172.21.15.19:41510) 2024-04-25T11:01:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1530283208 learned_addr learned my addr 172.21.15.19:0/1530283208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3daffd640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3dc102b60 0x7fa3dc199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.605+0000 7fa3db7fe640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3dc10d3a0 0x7fa3dc19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1530283208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3dc102b60 msgr2=0x7fa3dc199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3dc102b60 0x7fa3dc199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1530283208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3dc10d3a0 msgr2=0x7fa3dc19dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3dc10d3a0 0x7fa3dc19dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1530283208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc19e3e0 con 0x7fa3dc108a40 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc108a40 0x7fa3dc199ae0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d8ff9640 1 -- 172.21.15.19:0/1530283208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4017070 con 0x7fa3dc108a40 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d8ff9640 1 -- 172.21.15.19:0/1530283208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c4009c30 con 0x7fa3dc108a40 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc19e6d0 con 0x7fa3dc108a40 2024-04-25T11:01:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3d8ff9640 1 -- 172.21.15.19:0/1530283208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4003d10 con 0x7fa3dc108a40 2024-04-25T11:01:31.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.606+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc1a6050 con 0x7fa3dc108a40 2024-04-25T11:01:31.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.607+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc06a190 con 0x7fa3dc108a40 2024-04-25T11:01:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.608+0000 7fa3d8ff9640 1 -- 172.21.15.19:0/1530283208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa3c4003eb0 con 0x7fa3dc108a40 2024-04-25T11:01:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.609+0000 7fa3d8ff9640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa3c80afc60 0x7fa3c80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.609+0000 7fa3daffd640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa3c80afc60 0x7fa3c80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.609+0000 7fa3d8ff9640 1 -- 172.21.15.19:0/1530283208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fa3c40ded20 con 0x7fa3dc108a40 2024-04-25T11:01:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.610+0000 7fa3daffd640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa3c80afc60 0x7fa3c80b2170 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005fd0 tx=0x7fa3cc00b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.614+0000 7fa3d8ff9640 1 -- 172.21.15.19:0/1530283208 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3c4020030 con 0x7fa3dc108a40 2024-04-25T11:01:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.838+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3dc106f00 con 0x7fa3dc108a40 2024-04-25T11:01:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.839+0000 7fa3d8ff9640 1 -- 172.21.15.19:0/1530283208 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fa3c4009de0 con 0x7fa3dc108a40 2024-04-25T11:01:31.839 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:01:31.839 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:01:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:01:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa3c80afc60 msgr2=0x7fa3c80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa3c80afc60 0x7fa3c80b2170 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005fd0 tx=0x7fa3cc00b390 comp rx=0 tx=0).stop 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc108a40 msgr2=0x7fa3dc199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc108a40 0x7fa3dc199ae0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c400dde0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 shutdown_connections 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa3c80afc60 0x7fa3c80b2170 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3dc10d3a0 0x7fa3dc19dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3dc108a40 0x7fa3dc199ae0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 --2- 172.21.15.19:0/1530283208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3dc102b60 0x7fa3dc199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 >> 172.21.15.19:0/1530283208 conn(0x7fa3dc078040 msgr2=0x7fa3dc1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 shutdown_connections 2024-04-25T11:01:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.841+0000 7fa3e15ee640 1 -- 172.21.15.19:0/1530283208 wait complete. 2024-04-25T11:01:31.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:01:31.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.967+0000 7fbb4d449640 1 -- 172.21.15.19:0/3839535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 msgr2=0x7fbb48102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.967+0000 7fbb4d449640 1 --2- 172.21.15.19:0/3839535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb48102610 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009900 tx=0x7fbb3802a3f0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.967+0000 7fbb4d449640 1 -- 172.21.15.19:0/3839535675 shutdown_connections 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.967+0000 7fbb4d449640 1 --2- 172.21.15.19:0/3839535675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb481031c0 0x7fbb4810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.967+0000 7fbb4d449640 1 --2- 172.21.15.19:0/3839535675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb48108eb0 0x7fbb48102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.967+0000 7fbb4d449640 1 --2- 172.21.15.19:0/3839535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb48102610 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.968+0000 7fbb4d449640 1 -- 172.21.15.19:0/3839535675 >> 172.21.15.19:0/3839535675 conn(0x7fbb48078110 msgr2=0x7fbb48078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.968+0000 7fbb4d449640 1 -- 172.21.15.19:0/3839535675 shutdown_connections 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.968+0000 7fbb4d449640 1 -- 172.21.15.19:0/3839535675 wait complete. 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.968+0000 7fbb4d449640 1 Processor -- start 2024-04-25T11:01:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.968+0000 7fbb4d449640 1 -- start start 2024-04-25T11:01:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb481031c0 0x7fbb4819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb4819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb48108eb0 0x7fbb4819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48112c10 con 0x7fbb48108a10 2024-04-25T11:01:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48112a90 con 0x7fbb481031c0 2024-04-25T11:01:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48112d90 con 0x7fbb48108eb0 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb4819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb4819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41536/0 (socket says 172.21.15.19:41536) 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 -- 172.21.15.19:0/1498974151 learned_addr learned my addr 172.21.15.19:0/1498974151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb477fe640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb48108eb0 0x7fbb4819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb481031c0 0x7fbb4819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 -- 172.21.15.19:0/1498974151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb48108eb0 msgr2=0x7fbb4819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb48108eb0 0x7fbb4819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 -- 172.21.15.19:0/1498974151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb481031c0 msgr2=0x7fbb4819b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb481031c0 0x7fbb4819b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 -- 172.21.15.19:0/1498974151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb481a6800 con 0x7fbb48108a10 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb481031c0 0x7fbb4819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb477fe640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb48108eb0 0x7fbb4819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb4819ba90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c970 tx=0x7fbb3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb44ff9640 1 -- 172.21.15.19:0/1498974151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30007c70 con 0x7fbb48108a10 2024-04-25T11:01:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb44ff9640 1 -- 172.21.15.19:0/1498974151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb30007e10 con 0x7fbb48108a10 2024-04-25T11:01:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb481a6af0 con 0x7fbb48108a10 2024-04-25T11:01:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb44ff9640 1 -- 172.21.15.19:0/1498974151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300187b0 con 0x7fbb48108a10 2024-04-25T11:01:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.969+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb481a7090 con 0x7fbb48108a10 2024-04-25T11:01:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.972+0000 7fbb543d2740 1 Processor -- start 2024-04-25T11:01:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.972+0000 7fbb543d2740 1 -- start start 2024-04-25T11:01:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.972+0000 7fbb44ff9640 1 -- 172.21.15.19:0/1498974151 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fbb3000fab0 con 0x7fbb48108a10 2024-04-25T11:01:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 0x55f1aac676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa89d4d0 0x55f1aac67c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8a56b0 0x55f1aac78ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb543d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1aabccd80 con 0x55f1aa8a56b0 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb543d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f1aab86a20 con 0x55f1aa8c6390 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb543d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f1aabc8d40 con 0x55f1aa89d4d0 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8a56b0 0x55f1aac78ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8a56b0 0x55f1aac78ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41550/0 (socket says 172.21.15.19:41550) 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 -- 172.21.15.19:0/2498130109 learned_addr learned my addr 172.21.15.19:0/2498130109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb477fe640 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 0x55f1aac676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa89d4d0 0x55f1aac67c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 -- 172.21.15.19:0/2498130109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa89d4d0 msgr2=0x55f1aac67c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa89d4d0 0x55f1aac67c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 -- 172.21.15.19:0/2498130109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 msgr2=0x55f1aac676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 0x55f1aac676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 -- 172.21.15.19:0/2498130109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1aab91c30 con 0x55f1aa8a56b0 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa89d4d0 0x55f1aac67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb44ff9640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbb340afce0 0x7fbb340b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb477fe640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbb340afce0 0x7fbb340b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8a56b0 0x55f1aac78ab0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbb300dc6e0 tx=0x7fbb300e2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc528e8f2ee66c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.973+0000 7fbb44ff9640 1 -- 172.21.15.19:0/1498974151 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fbb300df020 con 0x7fbb48108a10 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.974+0000 7fbb3d7fa640 1 -- 172.21.15.19:0/2498130109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300a3080 con 0x55f1aa8a56b0 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.974+0000 7fbb3d7fa640 1 -- 172.21.15.19:0/2498130109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb300e7e50 con 0x55f1aa8a56b0 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.974+0000 7fbb3d7fa640 1 -- 172.21.15.19:0/2498130109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300a6930 con 0x55f1aa8a56b0 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 -- 172.21.15.19:0/2498130109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8a56b0 msgr2=0x55f1aac78ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb477fe640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbb340afce0 0x7fbb340b21f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbb48075eb0 tx=0x7fbb400095a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8a56b0 0x55f1aac78ab0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbb300dc6e0 tx=0x7fbb300e2f90 comp rx=0 tx=0).stop 2024-04-25T11:01:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 -- 172.21.15.19:0/2498130109 shutdown_connections 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8a56b0 0x55f1aac78ab0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa89d4d0 0x55f1aac67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2498130109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 0x55f1aac676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 -- 172.21.15.19:0/2498130109 >> 172.21.15.19:0/2498130109 conn(0x55f1aa8ba4e0 msgr2=0x55f1aac63fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 -- 172.21.15.19:0/2498130109 shutdown_connections 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 -- 172.21.15.19:0/2498130109 wait complete. 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 -- start start 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 Processor -- start 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac76630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac76b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.975+0000 7fbb543d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1aac791a0 con 0x55f1aa8c6390 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb543d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f1aab53dc0 con 0x55f1aa89d4d0 2024-04-25T11:01:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb543d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f1aac79320 con 0x55f1aa8a56b0 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb543d2740 10 client.0 _ll_register_callbacks cb 0x55f1aab6f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac826e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41562/0 (socket says 172.21.15.19:41562) 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 -- 172.21.15.19:0/579368460 learned_addr learned my addr 172.21.15.19:0/579368460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac76630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac76b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 -- 172.21.15.19:0/579368460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 msgr2=0x55f1aac76b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac76b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 -- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 msgr2=0x55f1aac76630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac76630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 -- 172.21.15.19:0/579368460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1aa8de3f0 con 0x55f1aa8c6390 2024-04-25T11:01:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac76b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb477fe640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac826e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbb40010910 tx=0x7fbb40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c900aa3b716c76c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/579368460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40004d20 con 0x55f1aa8c6390 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb300e7e50 con 0x55f1aa8c6390 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.976+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/579368460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fbb4000faf0 con 0x55f1aa8c6390 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/579368460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbb40010e90 con 0x55f1aa8c6390 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 10 client.16614 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f1aa9a87d0 con 0x55f1aa8c6390 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/579368460 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb300e7e50 con 0x55f1aa8c6390 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/579368460 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fbb4000f040 con 0x55f1aa8c6390 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 10 client.16614 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 20 client.16614 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa884f60 0x55f1aac88bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 4 client.16614 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:01:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1aabda990 con 0x55f1aa884f60 2024-04-25T11:01:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.977+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa884f60 0x55f1aac88bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.978+0000 7fbb2e7fc640 10 client.16614 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.978+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa884f60 0x55f1aac88bb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ff238f14f5d0f2 server_cookie=6458114e9251db5d in_seq=0 out_seq=0 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/579368460 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1aabda990 con 0x55f1aa884f60 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb2e7fc640 10 client.16614 handle_command_reply: tid=0 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 client.16614 shutdown 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 msgr2=0x55f1aac826e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac826e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbb40010910 tx=0x7fbb40004a40 comp rx=0 tx=0).stop 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 shutdown_connections 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa884f60 0x55f1aac88bb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac826e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac76b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 --2- 172.21.15.19:0/579368460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac76630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb2e7fc640 0 client.16614 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 >> 172.21.15.19:0/579368460 conn(0x55f1aa8ba4e0 msgr2=0x55f1aac694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 shutdown_connections 2024-04-25T11:01:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 1 -- 172.21.15.19:0/579368460 wait complete. 2024-04-25T11:01:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.979+0000 7fbb543d2740 20 client.16614 trim_cache size 0 max 16384 2024-04-25T11:01:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 Processor -- start 2024-04-25T11:01:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 -- start start 2024-04-25T11:01:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac8e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481ab890 con 0x55f1aa8c6390 2024-04-25T11:01:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481ae0f0 con 0x55f1aa89d4d0 2024-04-25T11:01:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb543d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481abee0 con 0x55f1aa8a56b0 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac8e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac8e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41568/0 (socket says 172.21.15.19:41568) 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb477fe640 1 -- 172.21.15.19:0/2383521328 learned_addr learned my addr 172.21.15.19:0/2383521328 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.993+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb477fe640 1 -- 172.21.15.19:0/2383521328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 msgr2=0x55f1aac8e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb477fe640 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac8e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb477fe640 1 -- 172.21.15.19:0/2383521328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 msgr2=0x55f1aac822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb477fe640 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb477fe640 1 -- 172.21.15.19:0/2383521328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1aac68d50 con 0x55f1aa8c6390 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb477fe640 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac8e8b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbb40005a70 tx=0x7fbb40010570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ca96428f763a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb3cff9640 1 -- 172.21.15.19:0/2383521328 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40036040 con 0x55f1aa8c6390 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb3cff9640 1 -- 172.21.15.19:0/2383521328 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4000f040 con 0x55f1aa8c6390 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.994+0000 7fbb3cff9640 1 -- 172.21.15.19:0/2383521328 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40030850 con 0x55f1aa8c6390 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- 172.21.15.19:0/2383521328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 msgr2=0x55f1aac8e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac8e8b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbb40005a70 tx=0x7fbb40010570 comp rx=0 tx=0).stop 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- 172.21.15.19:0/2383521328 shutdown_connections 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa8c6390 0x55f1aac8e8b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac8e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 --2- 172.21.15.19:0/2383521328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa89d4d0 0x55f1aac822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- 172.21.15.19:0/2383521328 >> 172.21.15.19:0/2383521328 conn(0x55f1aa8ba4e0 msgr2=0x55f1aac715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- 172.21.15.19:0/2383521328 shutdown_connections 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- 172.21.15.19:0/2383521328 wait complete. 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- start start 2024-04-25T11:01:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 Processor -- start 2024-04-25T11:01:32.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa89d4d0 0x55f1aac79600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:32.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac79b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:32.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 0x55f1aac7a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:32.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481ac5b0 con 0x55f1aa89d4d0 2024-04-25T11:01:32.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481ada20 con 0x55f1aa8c6390 2024-04-25T11:01:32.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.995+0000 7fbb543d2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481acc80 con 0x55f1aa8a56b0 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb543d2740 10 client.0 _ll_register_callbacks cb 0x7fbb40009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa89d4d0 0x55f1aac79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa89d4d0 0x55f1aac79600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41582/0 (socket says 172.21.15.19:41582) 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 -- 172.21.15.19:0/1054334659 learned_addr learned my addr 172.21.15.19:0/1054334659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac79b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 -- 172.21.15.19:0/1054334659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 msgr2=0x55f1aac79b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac79b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 -- 172.21.15.19:0/1054334659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 msgr2=0x55f1aac7a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 0x55f1aac7a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 -- 172.21.15.19:0/1054334659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1aaceef10 con 0x55f1aa89d4d0 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb46ffd640 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa89d4d0 0x55f1aac79600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbb380092a0 tx=0x7fbb38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe100b9726bf007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:32.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/1054334659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38035040 con 0x55f1aa89d4d0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/1054334659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fbb38005040 con 0x55f1aa89d4d0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4000f040 con 0x55f1aa89d4d0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/1054334659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbb38051050 con 0x55f1aa89d4d0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb543d2740 10 client.16620 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.996+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f1aacf0160 con 0x55f1aa89d4d0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/1054334659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4000f040 con 0x55f1aa89d4d0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/1054334659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fbb38042ce0 con 0x55f1aa89d4d0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb543d2740 10 client.16620 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb543d2740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb543d2740 20 client.16620 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb543d2740 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa85a3a0 0x55f1aacf2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb543d2740 4 client.16620 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1aabd4800 con 0x55f1aa85a3a0 2024-04-25T11:01:32.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.997+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa85a3a0 0x55f1aacf2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb2e7fc640 10 client.16620 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb3ffff640 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa85a3a0 0x55f1aacf2920 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525ddfbc58a17d13 server_cookie=702240f1f12622d in_seq=0 out_seq=0 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb2e7fc640 1 -- 172.21.15.19:0/1054334659 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1aabd4800 con 0x55f1aa85a3a0 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb2e7fc640 10 client.16620 handle_command_reply: tid=0 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 client.16620 shutdown 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa89d4d0 msgr2=0x55f1aac79600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa89d4d0 0x55f1aac79600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbb380092a0 tx=0x7fbb38004040 comp rx=0 tx=0).stop 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 shutdown_connections 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f1aa85a3a0 0x55f1aacf2920 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1aa8c6390 0x55f1aac7a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb2e7fc640 0 client.16620 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1aa8a56b0 0x55f1aac79b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 --2- 172.21.15.19:0/1054334659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1aa89d4d0 0x55f1aac79600 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 >> 172.21.15.19:0/1054334659 conn(0x55f1aa8ba4e0 msgr2=0x55f1aac74cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:32.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 shutdown_connections 2024-04-25T11:01:32.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 1 -- 172.21.15.19:0/1054334659 wait complete. 2024-04-25T11:01:32.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.998+0000 7fbb543d2740 20 client.16620 trim_cache size 0 max 16384 2024-04-25T11:01:32.005 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbb340afce0 msgr2=0x7fbb340b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbb340afce0 0x7fbb340b21f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbb48075eb0 tx=0x7fbb400095a0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 msgr2=0x7fbb4819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb4819ba90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c970 tx=0x7fbb3000ce30 comp rx=0 tx=0).stop 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 shutdown_connections 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fbb340afce0 0x7fbb340b21f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb48108eb0 0x7fbb4819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb48108a10 0x7fbb4819ba90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 --2- 172.21.15.19:0/1498974151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb481031c0 0x7fbb4819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 >> 172.21.15.19:0/1498974151 conn(0x7fbb48078110 msgr2=0x7fbb481040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 shutdown_connections 2024-04-25T11:01:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:31.999+0000 7fbb4d449640 1 -- 172.21.15.19:0/1498974151 wait complete. 2024-04-25T11:01:32.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:32 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1530283208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:01:32.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:32 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1530283208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:01:32.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:32 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1530283208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:01:33.011 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:01:33.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:01:33.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 -- 172.21.15.19:0/3921518133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109fc0 msgr2=0x7f50c8103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 --2- 172.21.15.19:0/3921518133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109fc0 0x7f50c8103c00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009900 tx=0x7f50b402a410 comp rx=0 tx=0).stop 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 -- 172.21.15.19:0/3921518133 shutdown_connections 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 --2- 172.21.15.19:0/3921518133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c8104270 0x7f50c8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 --2- 172.21.15.19:0/3921518133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109fc0 0x7f50c8103c00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 --2- 172.21.15.19:0/3921518133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c8109b20 0x7f50c81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 -- 172.21.15.19:0/3921518133 >> 172.21.15.19:0/3921518133 conn(0x7f50c80ff1e0 msgr2=0x7f50c8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 -- 172.21.15.19:0/3921518133 shutdown_connections 2024-04-25T11:01:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.142+0000 7f50cec27640 1 -- 172.21.15.19:0/3921518133 wait complete. 2024-04-25T11:01:33.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 Processor -- start 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 -- start start 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c8104270 0x7f50c819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109b20 0x7f50c819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c8109fc0 0x7f50c819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8113d10 con 0x7f50c8109b20 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8113b90 con 0x7f50c8104270 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.143+0000 7f50cec27640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8113e90 con 0x7f50c8109fc0 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109b20 0x7f50c819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109b20 0x7f50c819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47236/0 (socket says 172.21.15.19:47236) 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 -- 172.21.15.19:0/193043554 learned_addr learned my addr 172.21.15.19:0/193043554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c8109fc0 0x7f50c819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c8104270 0x7f50c819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 -- 172.21.15.19:0/193043554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c8109fc0 msgr2=0x7f50c819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c8109fc0 0x7f50c819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 -- 172.21.15.19:0/193043554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c8104270 msgr2=0x7f50c819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c8104270 0x7f50c819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 -- 172.21.15.19:0/193043554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c819f610 con 0x7f50c8109b20 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c8104270 0x7f50c819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c7fff640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109b20 0x7f50c819ad10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f50b402a930 tx=0x7f50b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c5ffb640 1 -- 172.21.15.19:0/193043554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b403f030 con 0x7f50c8109b20 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c5ffb640 1 -- 172.21.15.19:0/193043554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b4003aa0 con 0x7f50c8109b20 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50c5ffb640 1 -- 172.21.15.19:0/193043554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4035070 con 0x7f50c8109b20 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.144+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c819f8a0 con 0x7f50c8109b20 2024-04-25T11:01:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.145+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c819fde0 con 0x7f50c8109b20 2024-04-25T11:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.147+0000 7f50d5bbc740 1 Processor -- start 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.147+0000 7f50d5bbc740 1 -- start start 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.147+0000 7f50c5ffb640 1 -- 172.21.15.19:0/193043554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f50b4042030 con 0x7f50c8109b20 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770a87620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a87b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706c6730 0x559770a880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597709edb00 con 0x5597706e7410 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559770989920 con 0x5597706be550 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5597709e9ac0 con 0x5597706c6730 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770a87620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770a87620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47246/0 (socket says 172.21.15.19:47246) 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 -- 172.21.15.19:0/2014964213 learned_addr learned my addr 172.21.15.19:0/2014964213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a87b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50c7fff640 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706c6730 0x559770a880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 -- 172.21.15.19:0/2014964213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706c6730 msgr2=0x559770a880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706c6730 0x559770a880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 -- 172.21.15.19:0/2014964213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 msgr2=0x559770a87b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a87b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 -- 172.21.15.19:0/2014964213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597707c77e0 con 0x5597706e7410 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a87b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50c7fff640 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706c6730 0x559770a880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50c5ffb640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f50b80afce0 0x7f50b80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f50b80afce0 0x7f50b80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770a87620 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f50c0010370 tx=0x7f50c0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eee4aa621169213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.148+0000 7f50c5ffb640 1 -- 172.21.15.19:0/193043554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f50b40ff4c0 con 0x7f50c8109b20 2024-04-25T11:01:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.149+0000 7f50b2ffd640 1 -- 172.21.15.19:0/2014964213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0016070 con 0x5597706e7410 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.149+0000 7f50b2ffd640 1 -- 172.21.15.19:0/2014964213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c0004d20 con 0x5597706e7410 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.149+0000 7f50b2ffd640 1 -- 172.21.15.19:0/2014964213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0005350 con 0x5597706e7410 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 -- 172.21.15.19:0/2014964213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 msgr2=0x559770a87620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f50b80afce0 0x7f50b80b21f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x559770a88be0 tx=0x7f50bc00b420 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770a87620 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f50c0010370 tx=0x7f50c0010830 comp rx=0 tx=0).stop 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 -- 172.21.15.19:0/2014964213 shutdown_connections 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706c6730 0x559770a880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a87b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/2014964213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770a87620 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 -- 172.21.15.19:0/2014964213 >> 172.21.15.19:0/2014964213 conn(0x5597706db560 msgr2=0x559770a83dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 -- 172.21.15.19:0/2014964213 shutdown_connections 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 -- 172.21.15.19:0/2014964213 wait complete. 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 -- start start 2024-04-25T11:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.150+0000 7f50d5bbc740 1 Processor -- start 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a91c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a92180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706e7410 0x559770a9de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559770a94970 con 0x5597706c6730 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5597709875d0 con 0x5597706e7410 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559770a94af0 con 0x5597706be550 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 10 client.0 _ll_register_callbacks cb 0x559770975080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a92180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a92180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47262/0 (socket says 172.21.15.19:47262) 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 -- 172.21.15.19:0/88576683 learned_addr learned my addr 172.21.15.19:0/88576683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706e7410 0x559770a9de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a91c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 -- 172.21.15.19:0/88576683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 msgr2=0x559770a91c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a91c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 -- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706e7410 msgr2=0x559770a9de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706e7410 0x559770a9de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 -- 172.21.15.19:0/88576683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559770827f70 con 0x5597706c6730 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a91c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50c7fff640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a92180 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f50b40fcc70 tx=0x7f50b403b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c314334f3a1b9a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50a7fff640 1 -- 172.21.15.19:0/88576683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4049070 con 0x5597706c6730 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50a7fff640 1 -- 172.21.15.19:0/88576683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f50b40fce90 con 0x5597706c6730 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.151+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c0004d20 con 0x5597706c6730 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50a7fff640 1 -- 172.21.15.19:0/88576683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f50b40c7170 con 0x5597706c6730 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 10 client.16632 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5597706bf170 con 0x5597706c6730 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50a7fff640 1 -- 172.21.15.19:0/88576683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c0004d20 con 0x5597706c6730 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50a7fff640 1 -- 172.21.15.19:0/88576683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f50b40c38f0 con 0x5597706c6730 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 10 client.16632 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 20 client.16632 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5597706a61f0 0x559770aa3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 4 client.16632 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597709fb710 con 0x5597706a61f0 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.152+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5597706a61f0 0x559770aa3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.153+0000 7f50a7fff640 10 client.16632 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.153+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5597706a61f0 0x559770aa3e70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152cedf29cc036f8 server_cookie=fd6b24ad94c17a61 in_seq=0 out_seq=0 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50a7fff640 1 -- 172.21.15.19:0/88576683 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5597709fb710 con 0x5597706a61f0 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50a7fff640 10 client.16632 handle_command_reply: tid=0 2024-04-25T11:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 client.16632 shutdown 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 msgr2=0x559770a92180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a92180 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f50b40fcc70 tx=0x7f50b403b6b0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 shutdown_connections 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5597706a61f0 0x559770aa3e70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706e7410 0x559770a9de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a92180 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50a7fff640 0 client.16632 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/88576683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a91c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 >> 172.21.15.19:0/88576683 conn(0x5597706db560 msgr2=0x559770a87450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 shutdown_connections 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 1 -- 172.21.15.19:0/88576683 wait complete. 2024-04-25T11:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.154+0000 7f50d5bbc740 20 client.16632 trim_cache size 0 max 16384 2024-04-25T11:01:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 Processor -- start 2024-04-25T11:01:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 -- start start 2024-04-25T11:01:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a83e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706c6730 0x559770a9da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770aa9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81adb00 con 0x5597706e7410 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81b0360 con 0x5597706c6730 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81ae150 con 0x5597706be550 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770aa9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770aa9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47278/0 (socket says 172.21.15.19:47278) 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 -- 172.21.15.19:0/589082976 learned_addr learned my addr 172.21.15.19:0/589082976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50c7fff640 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706c6730 0x559770a9da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a83e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 -- 172.21.15.19:0/589082976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 msgr2=0x559770a83e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a83e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 -- 172.21.15.19:0/589082976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706c6730 msgr2=0x559770a9da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706c6730 0x559770a9da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 -- 172.21.15.19:0/589082976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559770a88c10 con 0x5597706e7410 2024-04-25T11:01:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.173+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770aa9640 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x559770a84790 tx=0x7f50c0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bf47cc190b7b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50b27fc640 1 -- 172.21.15.19:0/589082976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0016040 con 0x5597706e7410 2024-04-25T11:01:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50b27fc640 1 -- 172.21.15.19:0/589082976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c0005520 con 0x5597706e7410 2024-04-25T11:01:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50b27fc640 1 -- 172.21.15.19:0/589082976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c00127f0 con 0x5597706e7410 2024-04-25T11:01:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 -- 172.21.15.19:0/589082976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 msgr2=0x559770aa9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770aa9640 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x559770a84790 tx=0x7f50c0005c80 comp rx=0 tx=0).stop 2024-04-25T11:01:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 -- 172.21.15.19:0/589082976 shutdown_connections 2024-04-25T11:01:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706e7410 0x559770aa9640 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706c6730 0x559770a9da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/589082976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706be550 0x559770a83e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 -- 172.21.15.19:0/589082976 >> 172.21.15.19:0/589082976 conn(0x5597706db560 msgr2=0x559770a83710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.174+0000 7f50d5bbc740 1 -- 172.21.15.19:0/589082976 shutdown_connections 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 -- 172.21.15.19:0/589082976 wait complete. 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 -- start start 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 Processor -- start 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a9a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706e7410 0x559770a95db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81ae820 con 0x5597706c6730 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81afc90 con 0x5597706be550 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81aeef0 con 0x5597706e7410 2024-04-25T11:01:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.175+0000 7f50d5bbc740 10 client.0 _ll_register_callbacks cb 0x7f50b40c2e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a95760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47280/0 (socket says 172.21.15.19:47280) 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 -- 172.21.15.19:0/1268844528 learned_addr learned my addr 172.21.15.19:0/1268844528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a9a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706e7410 0x559770a95db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 -- 172.21.15.19:0/1268844528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706e7410 msgr2=0x559770a95db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706e7410 0x559770a95db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 -- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 msgr2=0x559770a9a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a9a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 -- 172.21.15.19:0/1268844528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559770a964f0 con 0x5597706c6730 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50cd1c4640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706e7410 0x559770a95db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a9a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50c7fff640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a95760 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f50b40c7360 tx=0x7f50b40c3ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6b8530c83e06ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50a7fff640 1 -- 172.21.15.19:0/1268844528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4049030 con 0x5597706c6730 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50a7fff640 1 -- 172.21.15.19:0/1268844528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f50b40d0240 con 0x5597706c6730 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c0005520 con 0x5597706c6730 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50a7fff640 1 -- 172.21.15.19:0/1268844528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f50b40c7ab0 con 0x5597706c6730 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50d5bbc740 10 client.16638 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559770b0c240 con 0x5597706c6730 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.176+0000 7f50a7fff640 1 -- 172.21.15.19:0/1268844528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c0005520 con 0x5597706c6730 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50a7fff640 1 -- 172.21.15.19:0/1268844528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f50b40ff040 con 0x5597706c6730 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50d5bbc740 10 client.16638 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50d5bbc740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50d5bbc740 20 client.16638 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55977067b630 0x559770b0e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50d5bbc740 4 client.16638 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5597709f5580 con 0x55977067b630 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55977067b630 0x559770b0e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50cc9c3640 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55977067b630 0x559770b0e840 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b4a257ed4d0af9 server_cookie=9847370ef5c4ff14 in_seq=0 out_seq=0 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.177+0000 7f50a7fff640 10 client.16638 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50a7fff640 1 -- 172.21.15.19:0/1268844528 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5597709f5580 con 0x55977067b630 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50a7fff640 10 client.16638 handle_command_reply: tid=0 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 client.16638 shutdown 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 msgr2=0x559770a95760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a95760 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f50b40c7360 tx=0x7f50b40c3ba0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 shutdown_connections 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55977067b630 0x559770b0e840 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5597706e7410 0x559770a95db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597706c6730 0x559770a95760 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 --2- 172.21.15.19:0/1268844528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597706be550 0x559770a9a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 >> 172.21.15.19:0/1268844528 conn(0x5597706db560 msgr2=0x559770a873b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50a7fff640 0 client.16638 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 shutdown_connections 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 1 -- 172.21.15.19:0/1268844528 wait complete. 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.179+0000 7f50d5bbc740 20 client.16638 trim_cache size 0 max 16384 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:01:33.185 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "f891fc3a-8cd2-466d-b98f-8cc290432c4b", 2024-04-25T11:01:33.186 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:01:33.186 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:01:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f50b80afce0 msgr2=0x7f50b80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f50b80afce0 0x7f50b80b21f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x559770a88be0 tx=0x7f50bc00b420 comp rx=0 tx=0).stop 2024-04-25T11:01:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109b20 msgr2=0x7f50c819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109b20 0x7f50c819ad10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f50b402a930 tx=0x7f50b4004270 comp rx=0 tx=0).stop 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 shutdown_connections 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f50b80afce0 0x7f50b80b21f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c8109fc0 0x7f50c819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c8109b20 0x7f50c819ad10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 --2- 172.21.15.19:0/193043554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c8104270 0x7f50c819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 >> 172.21.15.19:0/193043554 conn(0x7f50c80ff1e0 msgr2=0x7f50c810f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 shutdown_connections 2024-04-25T11:01:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.180+0000 7f50cec27640 1 -- 172.21.15.19:0/193043554 wait complete. 2024-04-25T11:01:33.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:01:33.192 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 -- 172.21.15.19:0/4183609613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d810d380 msgr2=0x7fa8d810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 --2- 172.21.15.19:0/4183609613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d810d380 0x7fa8d810f7a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009ca0 tx=0x7fa8d002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 -- 172.21.15.19:0/4183609613 shutdown_connections 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 --2- 172.21.15.19:0/4183609613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d810d380 0x7fa8d810f7a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 --2- 172.21.15.19:0/4183609613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d8102af0 0x7fa8d8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 --2- 172.21.15.19:0/4183609613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8d8108a20 0x7fa8d81025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 -- 172.21.15.19:0/4183609613 >> 172.21.15.19:0/4183609613 conn(0x7fa8d8078100 msgr2=0x7fa8d8078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 -- 172.21.15.19:0/4183609613 shutdown_connections 2024-04-25T11:01:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.306+0000 7fa8e0772640 1 -- 172.21.15.19:0/4183609613 wait complete. 2024-04-25T11:01:33.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 Processor -- start 2024-04-25T11:01:33.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 -- start start 2024-04-25T11:01:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8d8102af0 0x7fa8d819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d8108a20 0x7fa8d819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d810d380 0x7fa8d819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8112c00 con 0x7fa8d8108a20 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8112a80 con 0x7fa8d810d380 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8e0772640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8112d80 con 0x7fa8d8102af0 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8ddd0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d8108a20 0x7fa8d819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8ddd0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d8108a20 0x7fa8d819ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47298/0 (socket says 172.21.15.19:47298) 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8ddd0d640 1 -- 172.21.15.19:0/1029542568 learned_addr learned my addr 172.21.15.19:0/1029542568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8de50e640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8d8102af0 0x7fa8d819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.307+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d810d380 0x7fa8d819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8ddd0d640 1 -- 172.21.15.19:0/1029542568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8d8102af0 msgr2=0x7fa8d819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8d8102af0 0x7fa8d819b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8ddd0d640 1 -- 172.21.15.19:0/1029542568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d810d380 msgr2=0x7fa8d819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d810d380 0x7fa8d819bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8ddd0d640 1 -- 172.21.15.19:0/1029542568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d81a6850 con 0x7fa8d8108a20 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d810d380 0x7fa8d819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8de50e640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8d8102af0 0x7fa8d819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d8108a20 0x7fa8d819ba80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800d740 tx=0x7fa8c800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8cf7fe640 1 -- 172.21.15.19:0/1029542568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8017070 con 0x7fa8d8108a20 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d81a6b40 con 0x7fa8d8108a20 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8cf7fe640 1 -- 172.21.15.19:0/1029542568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c800de50 con 0x7fa8d8108a20 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8cf7fe640 1 -- 172.21.15.19:0/1029542568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8003c20 con 0x7fa8d8108a20 2024-04-25T11:01:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.308+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d81a7090 con 0x7fa8d8108a20 2024-04-25T11:01:33.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 Processor -- start 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 -- start start 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8cf7fe640 1 -- 172.21.15.19:0/1029542568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa8c8003dc0 con 0x7fa8d8108a20 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6478e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e64793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6485d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598e63dad80 con 0x5598e60d4390 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598e63948b0 con 0x5598e60b36b0 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8e7629740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5598e63d6d40 con 0x5598e60ab4d0 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8ded0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6478e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8ded0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6478e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47312/0 (socket says 172.21.15.19:47312) 2024-04-25T11:01:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8ded0f640 1 -- 172.21.15.19:0/3711735634 learned_addr learned my addr 172.21.15.19:0/3711735634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.311+0000 7fa8de50e640 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e64793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6485d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ded0f640 1 -- 172.21.15.19:0/3711735634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 msgr2=0x5598e64793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e64793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ded0f640 1 -- 172.21.15.19:0/3711735634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 msgr2=0x5598e6485d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6485d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ded0f640 1 -- 172.21.15.19:0/3711735634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e6365a40 con 0x5598e60d4390 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8cf7fe640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8c40afc60 0x7fa8c40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6478e70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa8d002f390 tx=0x7fa8d002f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2de3359e88c1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8de50e640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8c40afc60 0x7fa8c40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8cf7fe640 1 -- 172.21.15.19:0/1029542568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fa8c80deb20 con 0x7fa8d8108a20 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ccff9640 1 -- 172.21.15.19:0/3711735634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0031cf0 con 0x5598e60d4390 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ccff9640 1 -- 172.21.15.19:0/3711735634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d0036040 con 0x5598e60d4390 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.312+0000 7fa8ccff9640 1 -- 172.21.15.19:0/3711735634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d00042a0 con 0x5598e60d4390 2024-04-25T11:01:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 -- 172.21.15.19:0/3711735634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 msgr2=0x5598e6478e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8de50e640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8c40afc60 0x7fa8c40b2170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5598e647a640 tx=0x7fa8d400b550 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6478e70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa8d002f390 tx=0x7fa8d002f3c0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 -- 172.21.15.19:0/3711735634 shutdown_connections 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6485d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e64793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 --2- 172.21.15.19:0/3711735634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6478e70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 -- 172.21.15.19:0/3711735634 >> 172.21.15.19:0/3711735634 conn(0x5598e60c84e0 msgr2=0x5598e6475620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 -- 172.21.15.19:0/3711735634 shutdown_connections 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.313+0000 7fa8e7629740 1 -- 172.21.15.19:0/3711735634 wait complete. 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 -- start start 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 Processor -- start 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e6483960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6483ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60d4390 0x5598e648f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598e6486390 con 0x5598e60ab4d0 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598e6361d10 con 0x5598e60b36b0 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5598e6486510 con 0x5598e60d4390 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8e7629740 10 client.0 _ll_register_callbacks cb 0x5598e638e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e6483960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e6483960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47320/0 (socket says 172.21.15.19:47320) 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 -- 172.21.15.19:0/2834028556 learned_addr learned my addr 172.21.15.19:0/2834028556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6483ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60d4390 0x5598e648f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 -- 172.21.15.19:0/2834028556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60d4390 msgr2=0x5598e648f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60d4390 0x5598e648f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 -- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 msgr2=0x5598e6483ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6483ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.314+0000 7fa8de50e640 1 -- 172.21.15.19:0/2834028556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e61b67d0 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8de50e640 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e6483960 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4010730 tx=0x7fa8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa41377a96465ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/2834028556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400f990 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d0036040 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/2834028556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fa8d400fb30 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/2834028556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa8d400fd30 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 10 client.16650 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5598e60ec3f0 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/2834028556 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d0036040 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/2834028556 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa8d4027500 con 0x5598e60ab4d0 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 10 client.16650 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 20 client.16650 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e6092f60 0x5598e6495a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 4 client.16650 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.315+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598e63e8990 con 0x5598e6092f60 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.316+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e6092f60 0x5598e6495a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.316+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e6092f60 0x5598e6495a80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f31bd2df82f9af2 server_cookie=2ea7896eb7c1f898 in_seq=0 out_seq=0 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.316+0000 7fa8c1ffb640 10 client.16650 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/2834028556 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598e63e8990 con 0x5598e6092f60 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8c1ffb640 10 client.16650 handle_command_reply: tid=0 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 client.16650 shutdown 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 msgr2=0x5598e6483960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e6483960 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4010730 tx=0x7fa8d40046c0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 shutdown_connections 2024-04-25T11:01:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e6092f60 0x5598e6495a80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60d4390 0x5598e648f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6483ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 --2- 172.21.15.19:0/2834028556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e6483960 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8c1ffb640 0 client.16650 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 >> 172.21.15.19:0/2834028556 conn(0x5598e60c84e0 msgr2=0x5598e646f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 shutdown_connections 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 1 -- 172.21.15.19:0/2834028556 wait complete. 2024-04-25T11:01:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.317+0000 7fa8e7629740 20 client.16650 trim_cache size 0 max 16384 2024-04-25T11:01:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.335+0000 7fa8e7629740 1 Processor -- start 2024-04-25T11:01:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.335+0000 7fa8e7629740 1 -- start start 2024-04-25T11:01:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e64703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60b36b0 0x5598e64755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60d4390 0x5598e649b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8e7629740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81ab710 con 0x5598e60ab4d0 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8e7629740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81adf70 con 0x5598e60d4390 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8e7629740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81abd60 con 0x5598e60b36b0 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e64703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e64703f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47332/0 (socket says 172.21.15.19:47332) 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 -- 172.21.15.19:0/1883488141 learned_addr learned my addr 172.21.15.19:0/1883488141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60d4390 0x5598e649b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 -- 172.21.15.19:0/1883488141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60b36b0 msgr2=0x5598e64755a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60b36b0 0x5598e64755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60b36b0 0x5598e64755a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 -- 172.21.15.19:0/1883488141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60d4390 msgr2=0x5598e649b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60d4390 0x5598e649b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 -- 172.21.15.19:0/1883488141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e60ac0f0 con 0x5598e60ab4d0 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60d4390 0x5598e649b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60b36b0 0x5598e64755a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8de50e640 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e64703f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4027950 tx=0x7fa8d400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16c249b2000a8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.336+0000 7fa8c3fff640 1 -- 172.21.15.19:0/1883488141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4027b40 con 0x5598e60ab4d0 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8c3fff640 1 -- 172.21.15.19:0/1883488141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d4027ce0 con 0x5598e60ab4d0 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8c3fff640 1 -- 172.21.15.19:0/1883488141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4005550 con 0x5598e60ab4d0 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 -- 172.21.15.19:0/1883488141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 msgr2=0x5598e64703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e64703f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4027950 tx=0x7fa8d400bb40 comp rx=0 tx=0).stop 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 -- 172.21.15.19:0/1883488141 shutdown_connections 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60d4390 0x5598e649b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60b36b0 0x5598e64755a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 --2- 172.21.15.19:0/1883488141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60ab4d0 0x5598e64703f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 -- 172.21.15.19:0/1883488141 >> 172.21.15.19:0/1883488141 conn(0x5598e60c84e0 msgr2=0x5598e6484e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 -- 172.21.15.19:0/1883488141 shutdown_connections 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.337+0000 7fa8e7629740 1 -- 172.21.15.19:0/1883488141 wait complete. 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 -- start start 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 Processor -- start 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e6487330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6487870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6487db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81ac430 con 0x5598e60d4390 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81ad8a0 con 0x5598e60b36b0 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81acb00 con 0x5598e60ab4d0 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8e7629740 10 client.0 _ll_register_callbacks cb 0x7fa8d400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6487db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6487db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47338/0 (socket says 172.21.15.19:47338) 2024-04-25T11:01:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 -- 172.21.15.19:0/256878838 learned_addr learned my addr 172.21.15.19:0/256878838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8de50e640 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e6487330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ddd0d640 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6487870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 -- 172.21.15.19:0/256878838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 msgr2=0x5598e6487330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e6487330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 -- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 msgr2=0x5598e6487870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6487870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.338+0000 7fa8ded0f640 1 -- 172.21.15.19:0/256878838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e639b4b0 con 0x5598e60d4390 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8ded0f640 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6487db0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0004af0 tx=0x7fa8d0009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f17c106c4b57e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/256878838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0053070 con 0x5598e60d4390 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/256878838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fa8d0031bb0 con 0x5598e60d4390 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d4027ce0 con 0x5598e60d4390 2024-04-25T11:01:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/256878838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa8d005c050 con 0x5598e60d4390 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 10 client.16656 fetch_fsmap learned FSMap version 20 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5598e64fdc60 con 0x5598e60d4390 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/256878838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d4027ce0 con 0x5598e60d4390 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/256878838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa8d0056020 con 0x5598e60d4390 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 10 client.16656 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 20 client.16656 populate_metadata read hostname 'smithi019' 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e60683a0 0x5598e64fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 4 client.16656 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.339+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598e63e2800 con 0x5598e60683a0 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.340+0000 7fa8de50e640 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e60683a0 0x5598e64fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.340+0000 7fa8de50e640 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e60683a0 0x5598e64fffc0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3f2a8d690911fa server_cookie=e4f831d53144f5c in_seq=0 out_seq=0 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.340+0000 7fa8c1ffb640 10 client.16656 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.340+0000 7fa8c1ffb640 1 -- 172.21.15.19:0/256878838 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5598e63e2800 con 0x5598e60683a0 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.340+0000 7fa8c1ffb640 10 client.16656 handle_command_reply: tid=0 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 client.16656 shutdown 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 msgr2=0x5598e6487db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6487db0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0004af0 tx=0x7fa8d0009a10 comp rx=0 tx=0).stop 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 shutdown_connections 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5598e60683a0 0x5598e64fffc0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598e60d4390 0x5598e6487db0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598e60b36b0 0x5598e6487870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8c1ffb640 0 client.16656 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 --2- 172.21.15.19:0/256878838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598e60ab4d0 0x5598e6487330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 >> 172.21.15.19:0/256878838 conn(0x5598e60c84e0 msgr2=0x5598e6470150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 shutdown_connections 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 1 -- 172.21.15.19:0/256878838 wait complete. 2024-04-25T11:01:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e7629740 20 client.16656 trim_cache size 0 max 16384 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: "f891fc3a-8cd2-466d-b98f-8cc290432c4b": { 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "f891fc3a-8cd2-466d-b98f-8cc290432c4b", 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:01:33.348 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:01:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8c40afc60 msgr2=0x7fa8c40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8c40afc60 0x7fa8c40b2170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5598e647a640 tx=0x7fa8d400b550 comp rx=0 tx=0).stop 2024-04-25T11:01:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d8108a20 msgr2=0x7fa8d819ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d8108a20 0x7fa8d819ba80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800d740 tx=0x7fa8c800dc00 comp rx=0 tx=0).stop 2024-04-25T11:01:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 shutdown_connections 2024-04-25T11:01:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8c40afc60 0x7fa8c40b2170 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d810d380 0x7fa8d819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d8108a20 0x7fa8d819ba80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 --2- 172.21.15.19:0/1029542568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8d8102af0 0x7fa8d819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 >> 172.21.15.19:0/1029542568 conn(0x7fa8d8078100 msgr2=0x7fa8d81040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.341+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 shutdown_connections 2024-04-25T11:01:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:01:33.342+0000 7fa8e0772640 1 -- 172.21.15.19:0/1029542568 wait complete. 2024-04-25T11:01:33.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:33 smithi019 ceph-mon[29209]: pgmap v216: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:33.353 INFO:tasks.cephfs.filesystem:scrub status for tag:f891fc3a-8cd2-466d-b98f-8cc290432c4b - {'path': '/', 'tag': 'f891fc3a-8cd2-466d-b98f-8cc290432c4b', 'options': 'recursive,force'} 2024-04-25T11:01:33.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:33 smithi169 ceph-mon[33049]: pgmap v216: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:33.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:33 smithi112 ceph-mon[32799]: pgmap v216: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:34.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:34 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:01:34.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:34 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:34.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:34 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:01:34.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:34 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:34.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:34 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:01:34.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:34 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:34.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:34 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:01:34.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:34 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:34.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:34 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:01:34.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:34 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:34.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:34 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:01:34.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:34 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:01:35.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:35 smithi169 ceph-mon[33049]: pgmap v217: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:35.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:35 smithi112 ceph-mon[32799]: pgmap v217: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:35.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:35 smithi019 ceph-mon[29209]: pgmap v217: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:37 smithi169 ceph-mon[33049]: pgmap v218: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:37.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:37 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:01:37.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:37 smithi112 ceph-mon[32799]: pgmap v218: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:37.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:37 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:01:37.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:37 smithi019 ceph-mon[29209]: pgmap v218: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:37.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:37 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:01:39.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:39 smithi169 ceph-mon[33049]: pgmap v219: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:39.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:39 smithi112 ceph-mon[32799]: pgmap v219: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:39.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:39 smithi019 ceph-mon[29209]: pgmap v219: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:41.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:41 smithi169 ceph-mon[33049]: pgmap v220: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:41.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:41 smithi112 ceph-mon[32799]: pgmap v220: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:41.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:41 smithi019 ceph-mon[29209]: pgmap v220: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:42 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:01:42.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:42 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:01:42.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:42 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:01:43.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:43 smithi169 ceph-mon[33049]: pgmap v221: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:01:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:01:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:01:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:43 smithi112 ceph-mon[32799]: pgmap v221: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:01:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:01:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:01:43.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:43 smithi019 ceph-mon[29209]: pgmap v221: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:01:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:01:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:01:45.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:45 smithi169 ceph-mon[33049]: pgmap v222: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:45.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:45 smithi112 ceph-mon[32799]: pgmap v222: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:45.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:45 smithi019 ceph-mon[29209]: pgmap v222: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:01:47.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:47 smithi169 ceph-mon[33049]: pgmap v223: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:47.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:47 smithi112 ceph-mon[32799]: pgmap v223: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:47.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:47 smithi019 ceph-mon[29209]: pgmap v223: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:01:49.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:49 smithi169 ceph-mon[33049]: pgmap v224: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:49.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:49 smithi112 ceph-mon[32799]: pgmap v224: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:49.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:49 smithi019 ceph-mon[29209]: pgmap v224: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:51 smithi169 ceph-mon[33049]: pgmap v225: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:51.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:51 smithi112 ceph-mon[32799]: pgmap v225: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:51.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:51 smithi019 ceph-mon[29209]: pgmap v225: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:53.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:53 smithi169 ceph-mon[33049]: pgmap v226: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:53.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:53 smithi112 ceph-mon[32799]: pgmap v226: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:53.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:53 smithi019 ceph-mon[29209]: pgmap v226: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:55.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:55 smithi169 ceph-mon[33049]: pgmap v227: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:55.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:55 smithi112 ceph-mon[32799]: pgmap v227: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:55.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:55 smithi019 ceph-mon[29209]: pgmap v227: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:57 smithi169 ceph-mon[33049]: pgmap v228: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:57.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:57 smithi112 ceph-mon[32799]: pgmap v228: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:57.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:57 smithi019 ceph-mon[29209]: pgmap v228: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:01:59.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:01:59 smithi169 ceph-mon[33049]: pgmap v229: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:59.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:01:59 smithi112 ceph-mon[32799]: pgmap v229: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:01:59.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:01:59 smithi019 ceph-mon[29209]: pgmap v229: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:01.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:01 smithi169 ceph-mon[33049]: pgmap v230: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:01.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:01 smithi112 ceph-mon[32799]: pgmap v230: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:01.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:01 smithi019 ceph-mon[29209]: pgmap v230: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:03.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:02:03.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 -- 172.21.15.19:0/2104242929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 msgr2=0x7fed7c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 --2- 172.21.15.19:0/2104242929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c110640 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fed74009ca0 tx=0x7fed7402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 -- 172.21.15.19:0/2104242929 shutdown_connections 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 --2- 172.21.15.19:0/2104242929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c110640 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 --2- 172.21.15.19:0/2104242929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c109fc0 0x7fed7c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 --2- 172.21.15.19:0/2104242929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c109b20 0x7fed7c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 -- 172.21.15.19:0/2104242929 >> 172.21.15.19:0/2104242929 conn(0x7fed7c0ff1e0 msgr2=0x7fed7c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 -- 172.21.15.19:0/2104242929 shutdown_connections 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 -- 172.21.15.19:0/2104242929 wait complete. 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.484+0000 7fed817ee640 1 Processor -- start 2024-04-25T11:02:03.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed817ee640 1 -- start start 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed817ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed817ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c109b20 0x7fed7c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed817ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c109fc0 0x7fed7c076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed817ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c113d60 con 0x7fed7c104270 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed817ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c113be0 con 0x7fed7c109fc0 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed817ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c113ee0 con 0x7fed7c109b20 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45646/0 (socket says 172.21.15.19:45646) 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 -- 172.21.15.19:0/3583463060 learned_addr learned my addr 172.21.15.19:0/3583463060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c109fc0 0x7fed7c076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 -- 172.21.15.19:0/3583463060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c109b20 msgr2=0x7fed7c0764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7a7fc640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c109b20 0x7fed7c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c109b20 0x7fed7c0764f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 -- 172.21.15.19:0/3583463060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c109fc0 msgr2=0x7fed7c076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c109fc0 0x7fed7c076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7affd640 1 -- 172.21.15.19:0/3583463060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c1a89b0 con 0x7fed7c104270 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.485+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c109fc0 0x7fed7c076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.486+0000 7fed7affd640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c075fb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c970 tx=0x7fed6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.486+0000 7fed63fff640 1 -- 172.21.15.19:0/3583463060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c007bb0 con 0x7fed7c104270 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.486+0000 7fed63fff640 1 -- 172.21.15.19:0/3583463060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed6c007d50 con 0x7fed7c104270 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.486+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c1a8ca0 con 0x7fed7c104270 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.486+0000 7fed63fff640 1 -- 172.21.15.19:0/3583463060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00f450 con 0x7fed7c104270 2024-04-25T11:02:03.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.486+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c1a91a0 con 0x7fed7c104270 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed63fff640 1 -- 172.21.15.19:0/3583463060 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fed6c004f20 con 0x7fed7c104270 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 Processor -- start 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 -- start start 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0ab390 0x56543a44ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a44bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a4588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56543a3b1d80 con 0x56543a0ab390 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56543a36b8b0 con 0x56543a08a6b0 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed88775740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56543a3add40 con 0x56543a0824d0 2024-04-25T11:02:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0ab390 0x56543a44ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0ab390 0x56543a44ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45648/0 (socket says 172.21.15.19:45648) 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed7b7fe640 1 -- 172.21.15.19:0/91126825 learned_addr learned my addr 172.21.15.19:0/91126825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed7affd640 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a44bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed63fff640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fed640afc60 0x7fed640b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed7a7fc640 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a4588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed7b7fe640 1 -- 172.21.15.19:0/91126825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 msgr2=0x56543a44bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.489+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a44bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed7b7fe640 1 -- 172.21.15.19:0/91126825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 msgr2=0x56543a4588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a4588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed7b7fe640 1 -- 172.21.15.19:0/91126825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543a33ca40 con 0x56543a0ab390 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed7affd640 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a44bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed63fff640 1 -- 172.21.15.19:0/3583463060 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fed6c0df0c0 con 0x7fed7c104270 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fed640afc60 0x7fed640b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0ab390 0x56543a44ba60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fed740094e0 tx=0x7fed74031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ec1862c8ba0aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed617fa640 1 -- 172.21.15.19:0/91126825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74004040 con 0x56543a0ab390 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed617fa640 1 -- 172.21.15.19:0/91126825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed74036070 con 0x56543a0ab390 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.490+0000 7fed617fa640 1 -- 172.21.15.19:0/91126825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7402fa00 con 0x56543a0ab390 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 -- 172.21.15.19:0/91126825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0ab390 msgr2=0x56543a44ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0ab390 0x56543a44ba60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fed740094e0 tx=0x7fed74031af0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fed640afc60 0x7fed640b2170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fed7404b000 tx=0x7fed74048000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 -- 172.21.15.19:0/91126825 shutdown_connections 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a4588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a44bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 --2- 172.21.15.19:0/91126825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0ab390 0x56543a44ba60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 -- 172.21.15.19:0/91126825 >> 172.21.15.19:0/91126825 conn(0x56543a09f4e0 msgr2=0x56543a448210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 -- 172.21.15.19:0/91126825 shutdown_connections 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 -- 172.21.15.19:0/91126825 wait complete. 2024-04-25T11:02:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 -- start start 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.491+0000 7fed88775740 1 Processor -- start 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed88775740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0824d0 0x56543a456580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed88775740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a456ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed88775740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0ab390 0x56543a462310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed88775740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56543a458e90 con 0x56543a0824d0 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed88775740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56543a338d10 con 0x56543a08a6b0 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed88775740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56543a459010 con 0x56543a0ab390 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed88775740 10 client.0 _ll_register_callbacks cb 0x56543a365210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0824d0 0x56543a456580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0824d0 0x56543a456580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45664/0 (socket says 172.21.15.19:45664) 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 -- 172.21.15.19:0/352804752 learned_addr learned my addr 172.21.15.19:0/352804752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7a7fc640 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a456ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0ab390 0x56543a462310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 -- 172.21.15.19:0/352804752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0ab390 msgr2=0x56543a462310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0ab390 0x56543a462310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 -- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 msgr2=0x56543a456ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a456ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 -- 172.21.15.19:0/352804752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543a18d7d0 con 0x56543a0824d0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed7affd640 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0824d0 0x56543a456580 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0a7000 tx=0x7fed6c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f1fd731aa66575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.492+0000 7fed5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed5eff5640 1 -- 172.21.15.19:0/352804752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c01d040 con 0x56543a0824d0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed5eff5640 1 -- 172.21.15.19:0/352804752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fed6c0afcd0 con 0x56543a0824d0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed74036070 con 0x56543a0824d0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed5eff5640 1 -- 172.21.15.19:0/352804752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fed6c005410 con 0x56543a0824d0 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 10 client.16668 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56543a0c33f0 con 0x56543a0824d0 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed5eff5640 1 -- 172.21.15.19:0/352804752 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed74036070 con 0x56543a0824d0 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed5eff5640 1 -- 172.21.15.19:0/352804752 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fed6c0a2c70 con 0x56543a0824d0 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 10 client.16668 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 20 client.16668 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a069f60 0x56543a468580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 4 client.16668 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.493+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56543a3bf990 con 0x56543a069f60 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.494+0000 7fed7a7fc640 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a069f60 0x56543a468580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.494+0000 7fed5eff5640 10 client.16668 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.494+0000 7fed7a7fc640 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a069f60 0x56543a468580 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab4d70882524ea9 server_cookie=b378d3d2b554f947 in_seq=0 out_seq=0 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed5eff5640 1 -- 172.21.15.19:0/352804752 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56543a3bf990 con 0x56543a069f60 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed5eff5640 10 client.16668 handle_command_reply: tid=0 2024-04-25T11:02:03.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 client.16668 shutdown 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0824d0 msgr2=0x56543a456580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0824d0 0x56543a456580 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0a7000 tx=0x7fed6c0048f0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 shutdown_connections 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a069f60 0x56543a468580 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0ab390 0x56543a462310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed5eff5640 0 client.16668 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a08a6b0 0x56543a456ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 --2- 172.21.15.19:0/352804752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a0824d0 0x56543a456580 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 >> 172.21.15.19:0/352804752 conn(0x56543a09f4e0 msgr2=0x56543a44d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 shutdown_connections 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 1 -- 172.21.15.19:0/352804752 wait complete. 2024-04-25T11:02:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.495+0000 7fed88775740 20 client.16668 trim_cache size 0 max 16384 2024-04-25T11:02:03.503 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:03 smithi169 ceph-mon[33049]: pgmap v231: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:02:03.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 Processor -- start 2024-04-25T11:02:03.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 -- start start 2024-04-25T11:02:03.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a069f60 0x56543a450680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a0824d0 0x56543a450bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a45e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1afb60 con 0x56543a08a6b0 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b23c0 con 0x56543a0824d0 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed88775740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b01b0 con 0x56543a069f60 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a45e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a45e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45666/0 (socket says 172.21.15.19:45666) 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1039445667 learned_addr learned my addr 172.21.15.19:0/1039445667 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7a7fc640 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a0824d0 0x56543a450bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7affd640 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a069f60 0x56543a450680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1039445667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a069f60 msgr2=0x56543a450680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a069f60 0x56543a450680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1039445667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a0824d0 msgr2=0x56543a450bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a0824d0 0x56543a450bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.514+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1039445667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543a0830f0 con 0x56543a08a6b0 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.515+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a45e2e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fed740097b0 tx=0x7fed74009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1717a425c8af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.515+0000 7fed60ff9640 1 -- 172.21.15.19:0/1039445667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7402fcd0 con 0x56543a08a6b0 2024-04-25T11:02:03.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.515+0000 7fed60ff9640 1 -- 172.21.15.19:0/1039445667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed74036040 con 0x56543a08a6b0 2024-04-25T11:02:03.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.515+0000 7fed60ff9640 1 -- 172.21.15.19:0/1039445667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74004320 con 0x56543a08a6b0 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.515+0000 7fed88775740 1 -- 172.21.15.19:0/1039445667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 msgr2=0x56543a45e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.515+0000 7fed88775740 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a45e2e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fed740097b0 tx=0x7fed74009b20 comp rx=0 tx=0).stop 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.515+0000 7fed88775740 1 -- 172.21.15.19:0/1039445667 shutdown_connections 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a45e2e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a0824d0 0x56543a450bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 --2- 172.21.15.19:0/1039445667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a069f60 0x56543a450680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 -- 172.21.15.19:0/1039445667 >> 172.21.15.19:0/1039445667 conn(0x56543a09f4e0 msgr2=0x56543a455c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 -- 172.21.15.19:0/1039445667 shutdown_connections 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 -- 172.21.15.19:0/1039445667 wait complete. 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 -- start start 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 Processor -- start 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a069f60 0x56543a4c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a4c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a4596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b0880 con 0x56543a08a6b0 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b1cf0 con 0x56543a069f60 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b0f50 con 0x56543a0824d0 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed88775740 10 client.0 _ll_register_callbacks cb 0x7fed6c00c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a4596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a4596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45678/0 (socket says 172.21.15.19:45678) 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1254418828 learned_addr learned my addr 172.21.15.19:0/1254418828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.516+0000 7fed7a7fc640 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a4c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed7affd640 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a069f60 0x56543a4c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1254418828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 msgr2=0x56543a4c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a4c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a069f60 msgr2=0x56543a4c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:03.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a069f60 0x56543a4c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed7b7fe640 1 -- 172.21.15.19:0/1254418828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543a3724b0 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed7b7fe640 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a4596c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fed74031b30 tx=0x7fed7402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f21853ca6e022d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed5eff5640 1 -- 172.21.15.19:0/1254418828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740040b0 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed5eff5640 1 -- 172.21.15.19:0/1254418828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fed74058040 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed74036040 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed5eff5640 1 -- 172.21.15.19:0/1254418828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fed740042e0 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed88775740 10 client.16674 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.517+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56543a4d00c0 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed5eff5640 1 -- 172.21.15.19:0/1254418828 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed74036040 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed5eff5640 1 -- 172.21.15.19:0/1254418828 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fed74030c90 con 0x56543a08a6b0 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed88775740 10 client.16674 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed88775740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:03.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed88775740 20 client.16674 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed88775740 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a0ab390 0x56543a4d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed88775740 4 client.16674 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56543a3b9800 con 0x56543a0ab390 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.518+0000 7fed7affd640 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a0ab390 0x56543a4d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed5eff5640 10 client.16674 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed7affd640 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a0ab390 0x56543a4d2450 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9da0f2c8ecf9850 server_cookie=e60d2d33bdf4297f in_seq=0 out_seq=0 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed5eff5640 1 -- 172.21.15.19:0/1254418828 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56543a3b9800 con 0x56543a0ab390 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed5eff5640 10 client.16674 handle_command_reply: tid=0 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 client.16674 shutdown 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 msgr2=0x56543a4596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a4596c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fed74031b30 tx=0x7fed7402f840 comp rx=0 tx=0).stop 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 shutdown_connections 2024-04-25T11:02:03.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56543a0ab390 0x56543a4d2450 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56543a08a6b0 0x56543a4596c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56543a0824d0 0x56543a4c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 --2- 172.21.15.19:0/1254418828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56543a069f60 0x56543a4c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed5eff5640 0 client.16674 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 >> 172.21.15.19:0/1254418828 conn(0x56543a09f4e0 msgr2=0x56543a456630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 shutdown_connections 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 1 -- 172.21.15.19:0/1254418828 wait complete. 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.519+0000 7fed88775740 20 client.16674 trim_cache size 0 max 16384 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:02:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:02:03.526 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:02:03.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fed640afc60 msgr2=0x7fed640b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fed640afc60 0x7fed640b2170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fed7404b000 tx=0x7fed74048000 comp rx=0 tx=0).stop 2024-04-25T11:02:03.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 msgr2=0x7fed7c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c075fb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c970 tx=0x7fed6c00ce30 comp rx=0 tx=0).stop 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 shutdown_connections 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fed640afc60 0x7fed640b2170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed7c109fc0 0x7fed7c076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c109b20 0x7fed7c0764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 --2- 172.21.15.19:0/3583463060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed7c104270 0x7fed7c075fb0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 >> 172.21.15.19:0/3583463060 conn(0x7fed7c0ff1e0 msgr2=0x7fed7c106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 shutdown_connections 2024-04-25T11:02:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.520+0000 7fed817ee640 1 -- 172.21.15.19:0/3583463060 wait complete. 2024-04-25T11:02:03.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:03 smithi019 ceph-mon[29209]: pgmap v231: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:02:03.531 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:02:03.531 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:02:03.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:03 smithi112 ceph-mon[32799]: pgmap v231: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:02:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 -- 172.21.15.19:0/4017464190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c10e560 msgr2=0x7f999c110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 --2- 172.21.15.19:0/4017464190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c10e560 0x7f999c110980 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9994009ca0 tx=0x7f999402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 -- 172.21.15.19:0/4017464190 shutdown_connections 2024-04-25T11:02:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 --2- 172.21.15.19:0/4017464190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c10e560 0x7f999c110980 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 --2- 172.21.15.19:0/4017464190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c102060 0x7f999c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 --2- 172.21.15.19:0/4017464190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c109bb0 0x7f999c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 -- 172.21.15.19:0/4017464190 >> 172.21.15.19:0/4017464190 conn(0x7f999c0fd850 msgr2=0x7f999c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 -- 172.21.15.19:0/4017464190 shutdown_connections 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.643+0000 7f99a2677640 1 -- 172.21.15.19:0/4017464190 wait complete. 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 Processor -- start 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 -- start start 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c102060 0x7f999c102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c109bb0 0x7f999c103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c10e560 0x7f999c103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f999c113d20 con 0x7f999c102060 2024-04-25T11:02:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f999c113ba0 con 0x7f999c109bb0 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f99a2677640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f999c113ea0 con 0x7f999c10e560 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.644+0000 7f999bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c102060 0x7f999c102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c102060 0x7f999c102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45694/0 (socket says 172.21.15.19:45694) 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 -- 172.21.15.19:0/3585303192 learned_addr learned my addr 172.21.15.19:0/3585303192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f99a0c14640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c10e560 0x7f999c103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999b7fe640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c109bb0 0x7f999c103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 -- 172.21.15.19:0/3585303192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c10e560 msgr2=0x7f999c103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c10e560 0x7f999c103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 -- 172.21.15.19:0/3585303192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c109bb0 msgr2=0x7f999c103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c109bb0 0x7f999c103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 -- 172.21.15.19:0/3585303192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c1a8a20 con 0x7f999c102060 2024-04-25T11:02:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999b7fe640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c109bb0 0x7f999c103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f999bfff640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c102060 0x7f999c102d00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f998c00e9f0 tx=0x7f998c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f99997fa640 1 -- 172.21.15.19:0/3585303192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c00ccc0 con 0x7f999c102060 2024-04-25T11:02:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f99997fa640 1 -- 172.21.15.19:0/3585303192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998c0035f0 con 0x7f999c102060 2024-04-25T11:02:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c1a8d10 con 0x7f999c102060 2024-04-25T11:02:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f99997fa640 1 -- 172.21.15.19:0/3585303192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c00ce60 con 0x7f999c102060 2024-04-25T11:02:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.645+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1a91b0 con 0x7f999c102060 2024-04-25T11:02:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.646+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c06a190 con 0x7f999c102060 2024-04-25T11:02:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.648+0000 7f99997fa640 1 -- 172.21.15.19:0/3585303192 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f998c003aa0 con 0x7f999c102060 2024-04-25T11:02:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.648+0000 7f99997fa640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f99840afc60 0x7f99840b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.648+0000 7f999b7fe640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f99840afc60 0x7f99840b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.648+0000 7f99997fa640 1 -- 172.21.15.19:0/3585303192 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f998c0df4c0 con 0x7f999c102060 2024-04-25T11:02:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.649+0000 7f999b7fe640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f99840afc60 0x7f99840b2170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f999c100580 tx=0x7f9990005e50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:03.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.652+0000 7f99997fa640 1 -- 172.21.15.19:0/3585303192 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f998c0a7d60 con 0x7f999c102060 2024-04-25T11:02:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.878+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f999c104230 con 0x7f999c102060 2024-04-25T11:02:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.879+0000 7f99997fa640 1 -- 172.21.15.19:0/3585303192 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f998c0a7660 con 0x7f999c102060 2024-04-25T11:02:03.879 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:02:03.879 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:02:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:02:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.880+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f99840afc60 msgr2=0x7f99840b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.880+0000 7f99a2677640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f99840afc60 0x7f99840b2170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f999c100580 tx=0x7f9990005e50 comp rx=0 tx=0).stop 2024-04-25T11:02:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.880+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c102060 msgr2=0x7f999c102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.880+0000 7f99a2677640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c102060 0x7f999c102d00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f998c00e9f0 tx=0x7f998c00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 shutdown_connections 2024-04-25T11:02:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f99840afc60 0x7f99840b2170 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c10e560 0x7f999c103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c109bb0 0x7f999c103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 --2- 172.21.15.19:0/3585303192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f999c102060 0x7f999c102d00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 >> 172.21.15.19:0/3585303192 conn(0x7f999c0fd850 msgr2=0x7f999c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 shutdown_connections 2024-04-25T11:02:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:03.881+0000 7f99a2677640 1 -- 172.21.15.19:0/3585303192 wait complete. 2024-04-25T11:02:03.893 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 -- 172.21.15.19:0/3530463886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c108a10 msgr2=0x7f6f3c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 --2- 172.21.15.19:0/3530463886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c108a10 0x7f6f3c102610 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009900 tx=0x7f6f3802a410 comp rx=0 tx=0).stop 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 -- 172.21.15.19:0/3530463886 shutdown_connections 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 --2- 172.21.15.19:0/3530463886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f3c1031c0 0x7f6f3c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 --2- 172.21.15.19:0/3530463886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 --2- 172.21.15.19:0/3530463886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c108a10 0x7f6f3c102610 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 -- 172.21.15.19:0/3530463886 >> 172.21.15.19:0/3530463886 conn(0x7f6f3c078110 msgr2=0x7f6f3c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 -- 172.21.15.19:0/3530463886 shutdown_connections 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.007+0000 7f6f446d9640 1 -- 172.21.15.19:0/3530463886 wait complete. 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 Processor -- start 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 -- start start 2024-04-25T11:02:04.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c1031c0 0x7f6f3c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f3c108a10 0x7f6f3c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112c10 con 0x7f6f3c1031c0 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112a90 con 0x7f6f3c108eb0 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f446d9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112d90 con 0x7f6f3c108a10 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f42475640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c1031c0 0x7f6f3c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f42475640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c1031c0 0x7f6f3c19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45722/0 (socket says 172.21.15.19:45722) 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f42475640 1 -- 172.21.15.19:0/1219230434 learned_addr learned my addr 172.21.15.19:0/1219230434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f42c76640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f41c74640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f3c108a10 0x7f6f3c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f42475640 1 -- 172.21.15.19:0/1219230434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f3c108a10 msgr2=0x7f6f3c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.008+0000 7f6f42475640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f3c108a10 0x7f6f3c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f42475640 1 -- 172.21.15.19:0/1219230434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f3c108eb0 msgr2=0x7f6f3c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f42475640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f42475640 1 -- 172.21.15.19:0/1219230434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c19f650 con 0x7f6f3c1031c0 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f41c74640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f3c108a10 0x7f6f3c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f42c76640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f42475640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c1031c0 0x7f6f3c19a7f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802a930 tx=0x7f6f38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f337fe640 1 -- 172.21.15.19:0/1219230434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3803f030 con 0x7f6f3c1031c0 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f337fe640 1 -- 172.21.15.19:0/1219230434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f38003aa0 con 0x7f6f3c1031c0 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c19f8e0 con 0x7f6f3c1031c0 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f337fe640 1 -- 172.21.15.19:0/1219230434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38035070 con 0x7f6f3c1031c0 2024-04-25T11:02:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.009+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c19fe20 con 0x7f6f3c1031c0 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 Processor -- start 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 -- start start 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c08810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c08d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c156a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561389b6dd80 con 0x561389867390 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561389b27a20 con 0x56138983e4d0 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f4b581740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561389b69d40 con 0x5613898466b0 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f337fe640 1 -- 172.21.15.19:0/1219230434 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6f38042030 con 0x7f6f3c1031c0 2024-04-25T11:02:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f42c76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c08810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f42c76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c08810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45736/0 (socket says 172.21.15.19:45736) 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f42c76640 1 -- 172.21.15.19:0/1122751890 learned_addr learned my addr 172.21.15.19:0/1122751890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f41c74640 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.012+0000 7f6f42475640 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c08d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42c76640 1 -- 172.21.15.19:0/1122751890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 msgr2=0x561389c156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42c76640 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42c76640 1 -- 172.21.15.19:0/1122751890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 msgr2=0x561389c08d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42c76640 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c08d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42c76640 1 -- 172.21.15.19:0/1122751890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561389b32c30 con 0x561389867390 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42475640 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c08d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f337fe640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6f280afce0 0x7f6f280b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42c76640 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c08810 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009fb0 tx=0x7f6f34010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f2e51d9b75ffbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f42475640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6f280afce0 0x7f6f280b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f30ff9640 1 -- 172.21.15.19:0/1122751890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34016070 con 0x561389867390 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f30ff9640 1 -- 172.21.15.19:0/1122751890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f34012bc0 con 0x561389867390 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f30ff9640 1 -- 172.21.15.19:0/1122751890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3401bc50 con 0x561389867390 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.013+0000 7f6f337fe640 1 -- 172.21.15.19:0/1219230434 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f6f380ff530 con 0x7f6f3c1031c0 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 -- 172.21.15.19:0/1122751890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 msgr2=0x561389c08810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f42475640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6f280afce0 0x7f6f280b21f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f380fcd30 tx=0x7f6f38031a10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c08810 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009fb0 tx=0x7f6f34010800 comp rx=0 tx=0).stop 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 -- 172.21.15.19:0/1122751890 shutdown_connections 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c08d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 --2- 172.21.15.19:0/1122751890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c08810 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 -- 172.21.15.19:0/1122751890 >> 172.21.15.19:0/1122751890 conn(0x56138985b4e0 msgr2=0x561389c04fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:04.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 -- 172.21.15.19:0/1122751890 shutdown_connections 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.014+0000 7f6f4b581740 1 -- 172.21.15.19:0/1122751890 wait complete. 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 -- start start 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 Processor -- start 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c13780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c1f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561389c15d90 con 0x561389867390 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561389af4dc0 con 0x56138983e4d0 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561389c15f10 con 0x5613898466b0 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f4b581740 10 client.0 _ll_register_callbacks cb 0x561389b10600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f42c76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c1f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f42c76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c1f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45738/0 (socket says 172.21.15.19:45738) 2024-04-25T11:02:04.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f42c76640 1 -- 172.21.15.19:0/11212111 learned_addr learned my addr 172.21.15.19:0/11212111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f42c76640 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.015+0000 7f6f41c74640 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c13780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f42c76640 1 -- 172.21.15.19:0/11212111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 msgr2=0x561389c13780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f42c76640 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c13780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f42c76640 1 -- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 msgr2=0x561389c13240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f42c76640 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c13240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f42c76640 1 -- 172.21.15.19:0/11212111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138987f3f0 con 0x561389867390 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f42c76640 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c1f210 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6f34011010 tx=0x7f6f3401b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d112a1e5f83ee56e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f25ffb640 1 -- 172.21.15.19:0/11212111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3401f070 con 0x561389867390 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f25ffb640 1 -- 172.21.15.19:0/11212111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f6f34004c50 con 0x561389867390 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f34012bc0 con 0x561389867390 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f25ffb640 1 -- 172.21.15.19:0/11212111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6f34031050 con 0x561389867390 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f4b581740 10 client.16692 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:04.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5613899497d0 con 0x561389867390 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f25ffb640 1 -- 172.21.15.19:0/11212111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f34012bc0 con 0x561389867390 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.016+0000 7f6f25ffb640 1 -- 172.21.15.19:0/11212111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f6f3400f040 con 0x561389867390 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f4b581740 10 client.16692 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f4b581740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f4b581740 20 client.16692 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f4b581740 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389825f60 0x561389c254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f4b581740 4 client.16692 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561389b7b990 con 0x561389825f60 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f41c74640 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389825f60 0x561389c254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f25ffb640 10 client.16692 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.017+0000 7f6f41c74640 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389825f60 0x561389c254e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0e17c3063e6718 server_cookie=3d20a0b9ddae65c4 in_seq=0 out_seq=0 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f25ffb640 1 -- 172.21.15.19:0/11212111 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561389b7b990 con 0x561389825f60 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f25ffb640 10 client.16692 handle_command_reply: tid=0 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 client.16692 shutdown 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 msgr2=0x561389c1f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c1f210 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6f34011010 tx=0x7f6f3401b910 comp rx=0 tx=0).stop 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 shutdown_connections 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389825f60 0x561389c254e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561389867390 0x561389c1f210 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f25ffb640 0 client.16692 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c13780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 --2- 172.21.15.19:0/11212111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c13240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 >> 172.21.15.19:0/11212111 conn(0x56138985b4e0 msgr2=0x561389c0a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 shutdown_connections 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.018+0000 7f6f4b581740 1 -- 172.21.15.19:0/11212111 wait complete. 2024-04-25T11:02:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.019+0000 7f6f4b581740 20 client.16692 trim_cache size 0 max 16384 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.032+0000 7f6f4b581740 1 Processor -- start 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.032+0000 7f6f4b581740 1 -- start start 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561389825f60 0x561389c1f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613898466b0 0x561389c03d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f4b581740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1adb00 con 0x5613898466b0 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f4b581740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1b0360 con 0x56138983e4d0 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f4b581740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1ae150 con 0x561389825f60 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613898466b0 0x561389c03d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613898466b0 0x561389c03d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45748/0 (socket says 172.21.15.19:45748) 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 -- 172.21.15.19:0/2027841477 learned_addr learned my addr 172.21.15.19:0/2027841477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f41c74640 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561389825f60 0x561389c1f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 -- 172.21.15.19:0/2027841477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561389825f60 msgr2=0x561389c1f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561389825f60 0x561389c1f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 -- 172.21.15.19:0/2027841477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 msgr2=0x561389c127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 -- 172.21.15.19:0/2027841477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561389c13b10 con 0x5613898466b0 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613898466b0 0x561389c03d20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6f3401b8e0 tx=0x7f6f34010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b9e33adf044ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f27fff640 1 -- 172.21.15.19:0/2027841477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3401f020 con 0x5613898466b0 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f27fff640 1 -- 172.21.15.19:0/2027841477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f34010d00 con 0x5613898466b0 2024-04-25T11:02:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.033+0000 7f6f27fff640 1 -- 172.21.15.19:0/2027841477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34016040 con 0x5613898466b0 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 -- 172.21.15.19:0/2027841477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613898466b0 msgr2=0x561389c03d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613898466b0 0x561389c03d20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6f3401b8e0 tx=0x7f6f34010800 comp rx=0 tx=0).stop 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 -- 172.21.15.19:0/2027841477 shutdown_connections 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613898466b0 0x561389c03d20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56138983e4d0 0x561389c127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2027841477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561389825f60 0x561389c1f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 -- 172.21.15.19:0/2027841477 >> 172.21.15.19:0/2027841477 conn(0x56138985b4e0 msgr2=0x561389c03610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 -- 172.21.15.19:0/2027841477 shutdown_connections 2024-04-25T11:02:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.034+0000 7f6f4b581740 1 -- 172.21.15.19:0/2027841477 wait complete. 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 -- start start 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 Processor -- start 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561389825f60 0x561389c253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56138983e4d0 0x561389c258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1ae820 con 0x56138983e4d0 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1afc90 con 0x561389825f60 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1aeef0 con 0x5613898466b0 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f4b581740 10 client.0 _ll_register_callbacks cb 0x7f6f2c005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56138983e4d0 0x561389c258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56138983e4d0 0x561389c258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45752/0 (socket says 172.21.15.19:45752) 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 -- 172.21.15.19:0/2266465268 learned_addr learned my addr 172.21.15.19:0/2266465268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561389825f60 0x561389c253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 -- 172.21.15.19:0/2266465268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 msgr2=0x561389c25e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c25e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 -- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561389825f60 msgr2=0x561389c253b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561389825f60 0x561389c253b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.035+0000 7f6f41c74640 1 -- 172.21.15.19:0/2266465268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561389c8b830 con 0x56138983e4d0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f41c74640 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56138983e4d0 0x561389c258f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c10f920 tx=0x7f6f2c00f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fd8136f467cd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f25ffb640 1 -- 172.21.15.19:0/2266465268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c01c880 con 0x56138983e4d0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f25ffb640 1 -- 172.21.15.19:0/2266465268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f6f2c01ca20 con 0x56138983e4d0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f34010d00 con 0x56138983e4d0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f25ffb640 1 -- 172.21.15.19:0/2266465268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c02c050 con 0x56138983e4d0 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 10 client.16698 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561389c8cb10 con 0x56138983e4d0 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f25ffb640 1 -- 172.21.15.19:0/2266465268 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f34010d00 con 0x56138983e4d0 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f25ffb640 1 -- 172.21.15.19:0/2266465268 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f6f2c025460 con 0x56138983e4d0 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 10 client.16698 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 20 client.16698 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389867390 0x561389c8f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 4 client.16698 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.036+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561389b75800 con 0x561389867390 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389867390 0x561389c8f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f42c76640 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389867390 0x561389c8f280 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98f5f16011d907b server_cookie=3083123f983a79ea in_seq=0 out_seq=0 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f25ffb640 10 client.16698 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f25ffb640 1 -- 172.21.15.19:0/2266465268 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561389b75800 con 0x561389867390 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f25ffb640 10 client.16698 handle_command_reply: tid=0 2024-04-25T11:02:04.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 client.16698 shutdown 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56138983e4d0 msgr2=0x561389c258f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56138983e4d0 0x561389c258f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c10f920 tx=0x7f6f2c00f8b0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 shutdown_connections 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561389867390 0x561389c8f280 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613898466b0 0x561389c25e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f25ffb640 0 client.16698 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56138983e4d0 0x561389c258f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 --2- 172.21.15.19:0/2266465268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561389825f60 0x561389c253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 >> 172.21.15.19:0/2266465268 conn(0x56138985b4e0 msgr2=0x561389c0aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 shutdown_connections 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.037+0000 7f6f4b581740 1 -- 172.21.15.19:0/2266465268 wait complete. 2024-04-25T11:02:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f4b581740 20 client.16698 trim_cache size 0 max 16384 2024-04-25T11:02:04.045 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:02:04.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6f280afce0 msgr2=0x7f6f280b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6f280afce0 0x7f6f280b21f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f380fcd30 tx=0x7f6f38031a10 comp rx=0 tx=0).stop 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c1031c0 msgr2=0x7f6f3c19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c1031c0 0x7f6f3c19a7f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802a930 tx=0x7f6f38004270 comp rx=0 tx=0).stop 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 shutdown_connections 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6f280afce0 0x7f6f280b21f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f3c108a10 0x7f6f3c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 --2- 172.21.15.19:0/1219230434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f3c1031c0 0x7f6f3c19a7f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 >> 172.21.15.19:0/1219230434 conn(0x7f6f3c078110 msgr2=0x7f6f3c074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 shutdown_connections 2024-04-25T11:02:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:04.038+0000 7f6f446d9640 1 -- 172.21.15.19:0/1219230434 wait complete. 2024-04-25T11:02:04.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:04 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3585303192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:02:04.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:04 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3585303192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:02:04.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:04 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3585303192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:02:05.049 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:02:05.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:02:05.050 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:02:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 -- 172.21.15.19:0/1898247810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 msgr2=0x7f4314110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 --2- 172.21.15.19:0/1898247810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f4314110640 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f430c009ca0 tx=0x7f430c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 -- 172.21.15.19:0/1898247810 shutdown_connections 2024-04-25T11:02:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 --2- 172.21.15.19:0/1898247810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f4314110640 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 --2- 172.21.15.19:0/1898247810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4314109fc0 0x7f4314103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 --2- 172.21.15.19:0/1898247810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4314109b20 0x7f43141036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 -- 172.21.15.19:0/1898247810 >> 172.21.15.19:0/1898247810 conn(0x7f43140ff1e0 msgr2=0x7f4314101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 -- 172.21.15.19:0/1898247810 shutdown_connections 2024-04-25T11:02:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.182+0000 7f4318f72640 1 -- 172.21.15.19:0/1898247810 wait complete. 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 Processor -- start 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 -- start start 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f431419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4314109b20 0x7f431419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4314109fc0 0x7f431419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113d60 con 0x7f4314104270 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113be0 con 0x7f4314109fc0 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4318f72640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113ee0 con 0x7f4314109b20 2024-04-25T11:02:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f431419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f431419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45766/0 (socket says 172.21.15.19:45766) 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f431259c640 1 -- 172.21.15.19:0/1604141053 learned_addr learned my addr 172.21.15.19:0/1604141053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4311d9b640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4314109b20 0x7f431419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.183+0000 7f4312d9d640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4314109fc0 0x7f431419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f431259c640 1 -- 172.21.15.19:0/1604141053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4314109b20 msgr2=0x7f431419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f431259c640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4314109b20 0x7f431419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f431259c640 1 -- 172.21.15.19:0/1604141053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4314109fc0 msgr2=0x7f431419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f431259c640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4314109fc0 0x7f431419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f431259c640 1 -- 172.21.15.19:0/1604141053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431419f500 con 0x7f4314104270 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f4312d9d640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4314109fc0 0x7f431419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f4311d9b640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4314109b20 0x7f431419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f431259c640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f431419a720 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f430400d7a0 tx=0x7f430400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f43037fe640 1 -- 172.21.15.19:0/1604141053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304017070 con 0x7f4314104270 2024-04-25T11:02:05.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f43037fe640 1 -- 172.21.15.19:0/1604141053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4304009c30 con 0x7f4314104270 2024-04-25T11:02:05.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431419f7f0 con 0x7f4314104270 2024-04-25T11:02:05.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f43037fe640 1 -- 172.21.15.19:0/1604141053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400f620 con 0x7f4314104270 2024-04-25T11:02:05.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.184+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141a71f0 con 0x7f4314104270 2024-04-25T11:02:05.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.187+0000 7f431fe23740 1 Processor -- start 2024-04-25T11:02:05.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.187+0000 7f431fe23740 1 -- start start 2024-04-25T11:02:05.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f43037fe640 1 -- 172.21.15.19:0/1604141053 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f430401a070 con 0x7f4314104270 2024-04-25T11:02:05.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431fe23740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272f6f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431fe23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f6fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431fe23740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bae730 0x565272f700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431fe23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565272ed5b00 con 0x565272ba6550 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431fe23740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565272e71920 con 0x565272bae730 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431fe23740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x565272ed1ac0 con 0x565272bcf410 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f6fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f6fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45772/0 (socket says 172.21.15.19:45772) 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 -- 172.21.15.19:0/3491646893 learned_addr learned my addr 172.21.15.19:0/3491646893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f4311d9b640 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bae730 0x565272f700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f4312d9d640 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272f6f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 -- 172.21.15.19:0/3491646893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 msgr2=0x565272f6f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272f6f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 -- 172.21.15.19:0/3491646893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bae730 msgr2=0x565272f700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bae730 0x565272f700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f431259c640 1 -- 172.21.15.19:0/3491646893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565272caf7e0 con 0x565272ba6550 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.188+0000 7f4312d9d640 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272f6f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.189+0000 7f43037fe640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42fc0afce0 0x7f42fc0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.189+0000 7f431259c640 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f6fb60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f43040dc1e0 tx=0x7f43040e7660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d566e36d59e72d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.189+0000 7f4312d9d640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42fc0afce0 0x7f42fc0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.189+0000 7f43037fe640 1 -- 172.21.15.19:0/1604141053 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f43040deb20 con 0x7f4314104270 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.189+0000 7f4300ff9640 1 -- 172.21.15.19:0/3491646893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040a2b80 con 0x565272ba6550 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.189+0000 7f4300ff9640 1 -- 172.21.15.19:0/3491646893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43040e3040 con 0x565272ba6550 2024-04-25T11:02:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.189+0000 7f4300ff9640 1 -- 172.21.15.19:0/3491646893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040a6430 con 0x565272ba6550 2024-04-25T11:02:05.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 -- 172.21.15.19:0/3491646893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 msgr2=0x565272f6fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f6fb60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f43040dc1e0 tx=0x7f43040e7660 comp rx=0 tx=0).stop 2024-04-25T11:02:05.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f4312d9d640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42fc0afce0 0x7f42fc0b21f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c02f3e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 -- 172.21.15.19:0/3491646893 shutdown_connections 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bae730 0x565272f700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f6fb60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 --2- 172.21.15.19:0/3491646893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272f6f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 -- 172.21.15.19:0/3491646893 >> 172.21.15.19:0/3491646893 conn(0x565272bc3560 msgr2=0x565272f6bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 -- 172.21.15.19:0/3491646893 shutdown_connections 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 -- 172.21.15.19:0/3491646893 wait complete. 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 -- start start 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.190+0000 7f431fe23740 1 Processor -- start 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431fe23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f79b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431fe23740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f7a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431fe23740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f85e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431fe23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565272f7c970 con 0x565272ba6550 2024-04-25T11:02:05.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431fe23740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565272e6f5d0 con 0x565272bcf410 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431fe23740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x565272f7caf0 con 0x565272bae730 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431fe23740 10 client.0 _ll_register_callbacks cb 0x565272e5d080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f79b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f79b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45780/0 (socket says 172.21.15.19:45780) 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 -- 172.21.15.19:0/2764486746 learned_addr learned my addr 172.21.15.19:0/2764486746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f4312d9d640 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f85e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 -- 172.21.15.19:0/2764486746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 msgr2=0x565272f7a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f7a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 -- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 msgr2=0x565272f85e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f85e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 -- 172.21.15.19:0/2764486746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565272d0ff70 con 0x565272ba6550 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f431259c640 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f79b90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f43040dc320 tx=0x7f43040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685bdb152baac65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.191+0000 7f42f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f42f9ffb640 1 -- 172.21.15.19:0/2764486746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304021070 con 0x565272ba6550 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f42f9ffb640 1 -- 172.21.15.19:0/2764486746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f43040a2f50 con 0x565272ba6550 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43040e3040 con 0x565272ba6550 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f42f9ffb640 1 -- 172.21.15.19:0/2764486746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f43040a6610 con 0x565272ba6550 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 10 client.16710 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x565272ba7170 con 0x565272ba6550 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f42f9ffb640 1 -- 172.21.15.19:0/2764486746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43040e3040 con 0x565272ba6550 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f42f9ffb640 1 -- 172.21.15.19:0/2764486746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f43040a6f20 con 0x565272ba6550 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 10 client.16710 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 20 client.16710 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b8e1f0 0x565272f8bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 4 client.16710 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.192+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565272ee3710 con 0x565272b8e1f0 2024-04-25T11:02:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.193+0000 7f4311d9b640 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b8e1f0 0x565272f8bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.193+0000 7f4311d9b640 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b8e1f0 0x565272f8bdb0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d36caf7bbb55c5f server_cookie=d021393d29e8289c in_seq=0 out_seq=0 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.193+0000 7f42f9ffb640 10 client.16710 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f42f9ffb640 1 -- 172.21.15.19:0/2764486746 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565272ee3710 con 0x565272b8e1f0 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f42f9ffb640 10 client.16710 handle_command_reply: tid=0 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 client.16710 shutdown 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 msgr2=0x565272f79b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f79b90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f43040dc320 tx=0x7f43040057f0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 shutdown_connections 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b8e1f0 0x565272f8bdb0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f85e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f7a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f42f9ffb640 0 client.16710 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 --2- 172.21.15.19:0/2764486746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f79b90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 >> 172.21.15.19:0/2764486746 conn(0x565272bc3560 msgr2=0x565272f71340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 shutdown_connections 2024-04-25T11:02:05.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.194+0000 7f431fe23740 1 -- 172.21.15.19:0/2764486746 wait complete. 2024-04-25T11:02:05.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.195+0000 7f431fe23740 20 client.16710 trim_cache size 0 max 16384 2024-04-25T11:02:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 Processor -- start 2024-04-25T11:02:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 -- start start 2024-04-25T11:02:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f6aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f6fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ad9b0 con 0x565272ba6550 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43141b0230 con 0x565272bcf410 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431fe23740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ae020 con 0x565272bae730 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45786/0 (socket says 172.21.15.19:45786) 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.213+0000 7f431259c640 1 -- 172.21.15.19:0/222415835 learned_addr learned my addr 172.21.15.19:0/222415835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f4311d9b640 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f6aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f4312d9d640 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f6fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f431259c640 1 -- 172.21.15.19:0/222415835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 msgr2=0x565272f6aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f431259c640 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f6aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f431259c640 1 -- 172.21.15.19:0/222415835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 msgr2=0x565272f6fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f431259c640 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f6fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f431259c640 1 -- 172.21.15.19:0/222415835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565272f788e0 con 0x565272ba6550 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f4312d9d640 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f6fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f4311d9b640 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f6aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f431259c640 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f742f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f43040a2ba0 tx=0x7f43040dc260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb5287b6a17a592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f42fbfff640 1 -- 172.21.15.19:0/222415835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304021070 con 0x565272ba6550 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f42fbfff640 1 -- 172.21.15.19:0/222415835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43040a67d0 con 0x565272ba6550 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.214+0000 7f42fbfff640 1 -- 172.21.15.19:0/222415835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040a6a90 con 0x565272ba6550 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 -- 172.21.15.19:0/222415835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 msgr2=0x565272f742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f742f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f43040a2ba0 tx=0x7f43040dc260 comp rx=0 tx=0).stop 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 -- 172.21.15.19:0/222415835 shutdown_connections 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272bcf410 0x565272f6fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bae730 0x565272f6aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 --2- 172.21.15.19:0/222415835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272ba6550 0x565272f742f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 -- 172.21.15.19:0/222415835 >> 172.21.15.19:0/222415835 conn(0x565272bc3560 msgr2=0x565272f6f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 -- 172.21.15.19:0/222415835 shutdown_connections 2024-04-25T11:02:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 -- 172.21.15.19:0/222415835 wait complete. 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 -- start start 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.215+0000 7f431fe23740 1 Processor -- start 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272ba6550 0x565272f8c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272bae730 0x565272f8c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272ff1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ae6f0 con 0x565272bae730 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43141afb60 con 0x565272ba6550 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43141aedc0 con 0x565272bcf410 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 10 client.0 _ll_register_callbacks cb 0x7f43040e2bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272bae730 0x565272f8c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272bae730 0x565272f8c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45792/0 (socket says 172.21.15.19:45792) 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 -- 172.21.15.19:0/237080323 learned_addr learned my addr 172.21.15.19:0/237080323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431259c640 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272ba6550 0x565272f8c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 -- 172.21.15.19:0/237080323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 msgr2=0x565272ff1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272ff1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 -- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272ba6550 msgr2=0x565272f8c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272ba6550 0x565272f8c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 -- 172.21.15.19:0/237080323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565272ff2410 con 0x565272bae730 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f4311d9b640 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272bae730 0x565272f8c5a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f43140ff780 tx=0x7f43080035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef195a9d16f18c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f42f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f42f9ffb640 1 -- 172.21.15.19:0/237080323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308019250 con 0x565272bae730 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f42f9ffb640 1 -- 172.21.15.19:0/237080323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7f43080193f0 con 0x565272bae730 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.216+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43040a67d0 con 0x565272bae730 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f42f9ffb640 1 -- 172.21.15.19:0/237080323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4308029050 con 0x565272bae730 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 10 client.16716 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x565272ff3550 con 0x565272bae730 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f42f9ffb640 1 -- 172.21.15.19:0/237080323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43040a67d0 con 0x565272bae730 2024-04-25T11:02:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f42f9ffb640 1 -- 172.21.15.19:0/237080323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4308010ad0 con 0x565272bae730 2024-04-25T11:02:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 10 client.16716 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 20 client.16716 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b63630 0x565272ff5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 4 client.16716 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:02:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565272edd580 con 0x565272b63630 2024-04-25T11:02:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.217+0000 7f431259c640 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b63630 0x565272ff5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.218+0000 7f431259c640 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b63630 0x565272ff5cc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e4eb22c6809d2a server_cookie=4a13746f5e360950 in_seq=0 out_seq=0 2024-04-25T11:02:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.218+0000 7f42f9ffb640 10 client.16716 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f42f9ffb640 1 -- 172.21.15.19:0/237080323 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565272edd580 con 0x565272b63630 2024-04-25T11:02:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f42f9ffb640 10 client.16716 handle_command_reply: tid=0 2024-04-25T11:02:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 client.16716 shutdown 2024-04-25T11:02:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272bae730 msgr2=0x565272f8c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272bae730 0x565272f8c5a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f43140ff780 tx=0x7f43080035f0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 shutdown_connections 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x565272b63630 0x565272ff5cc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565272bcf410 0x565272ff1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f42f9ffb640 0 client.16716 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565272bae730 0x565272f8c5a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 --2- 172.21.15.19:0/237080323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565272ba6550 0x565272f8c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 >> 172.21.15.19:0/237080323 conn(0x565272bc3560 msgr2=0x565272f6ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 shutdown_connections 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 1 -- 172.21.15.19:0/237080323 wait complete. 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.219+0000 7f431fe23740 20 client.16716 trim_cache size 0 max 16384 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:02:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:02:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "28de13c3-e3eb-4a67-b01f-bc9826a15d29", 2024-04-25T11:02:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:02:05.225 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:02:05.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42fc0afce0 msgr2=0x7f42fc0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42fc0afce0 0x7f42fc0b21f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c02f3e0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 msgr2=0x7f431419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f431419a720 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f430400d7a0 tx=0x7f430400dc60 comp rx=0 tx=0).stop 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 shutdown_connections 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f42fc0afce0 0x7f42fc0b21f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4314109fc0 0x7f431419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4314109b20 0x7f431419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 --2- 172.21.15.19:0/1604141053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4314104270 0x7f431419a720 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 >> 172.21.15.19:0/1604141053 conn(0x7f43140ff1e0 msgr2=0x7f43141064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 shutdown_connections 2024-04-25T11:02:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.220+0000 7f4318f72640 1 -- 172.21.15.19:0/1604141053 wait complete. 2024-04-25T11:02:05.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:02:05.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:02:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 -- 172.21.15.19:0/563442643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 msgr2=0x7fb05c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 --2- 172.21.15.19:0/563442643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 0x7fb05c110630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb054009ca0 tx=0x7fb05402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 -- 172.21.15.19:0/563442643 shutdown_connections 2024-04-25T11:02:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 --2- 172.21.15.19:0/563442643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 0x7fb05c110630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 --2- 172.21.15.19:0/563442643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb05c109fb0 0x7fb05c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 --2- 172.21.15.19:0/563442643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 0x7fb05c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 -- 172.21.15.19:0/563442643 >> 172.21.15.19:0/563442643 conn(0x7fb05c0ff230 msgr2=0x7fb05c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 -- 172.21.15.19:0/563442643 shutdown_connections 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.346+0000 7fb062ebd640 1 -- 172.21.15.19:0/563442643 wait complete. 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 Processor -- start 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 -- start start 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 0x7fb05c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 0x7fb05c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb05c109fb0 0x7fb05c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c113d20 con 0x7fb05c109b10 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c113ba0 con 0x7fb05c104260 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb062ebd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c113ea0 con 0x7fb05c109fb0 2024-04-25T11:02:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 0x7fb05c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 0x7fb05c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45804/0 (socket says 172.21.15.19:45804) 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb060c59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 0x7fb05c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 -- 172.21.15.19:0/52418951 learned_addr learned my addr 172.21.15.19:0/52418951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb06145a640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb05c109fb0 0x7fb05c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 -- 172.21.15.19:0/52418951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb05c109fb0 msgr2=0x7fb05c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb05c109fb0 0x7fb05c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 -- 172.21.15.19:0/52418951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 msgr2=0x7fb05c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 0x7fb05c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 -- 172.21.15.19:0/52418951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c19f560 con 0x7fb05c109b10 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.347+0000 7fb05bfff640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 0x7fb05c19ac60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb04800e9f0 tx=0x7fb04800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.348+0000 7fb059ffb640 1 -- 172.21.15.19:0/52418951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04800ccc0 con 0x7fb05c109b10 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.348+0000 7fb059ffb640 1 -- 172.21.15.19:0/52418951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0480035f0 con 0x7fb05c109b10 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.348+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c19f850 con 0x7fb05c109b10 2024-04-25T11:02:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.348+0000 7fb059ffb640 1 -- 172.21.15.19:0/52418951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04800ce60 con 0x7fb05c109b10 2024-04-25T11:02:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.348+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c19fd50 con 0x7fb05c109b10 2024-04-25T11:02:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.350+0000 7fb059ffb640 1 -- 172.21.15.19:0/52418951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb048003aa0 con 0x7fb05c109b10 2024-04-25T11:02:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 Processor -- start 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 -- start start 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7994ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d7994efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d7995b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d798b4d80 con 0x555d795854d0 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d7986e8b0 con 0x555d7958d6b0 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb069d73740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555d798b0d40 con 0x555d795ae390 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d7994efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d7994efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45820/0 (socket says 172.21.15.19:45820) 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 -- 172.21.15.19:0/3628361078 learned_addr learned my addr 172.21.15.19:0/3628361078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb05bfff640 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d7995b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb06145a640 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7994ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb059ffb640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb04c0afc60 0x7fb04c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 -- 172.21.15.19:0/3628361078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 msgr2=0x555d7994ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7994ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 -- 172.21.15.19:0/3628361078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 msgr2=0x555d7995b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d7995b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb060c59640 1 -- 172.21.15.19:0/3628361078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7983fa40 con 0x555d795854d0 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.351+0000 7fb06145a640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb04c0afc60 0x7fb04c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.352+0000 7fb059ffb640 1 -- 172.21.15.19:0/52418951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fb0480df3f0 con 0x7fb05c109b10 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.352+0000 7fb060c59640 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d7994efa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb05000bcb0 tx=0x7fb0500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb0097cf5c1168e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.352+0000 7fb046ffd640 1 -- 172.21.15.19:0/3628361078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050004030 con 0x555d795854d0 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.352+0000 7fb046ffd640 1 -- 172.21.15.19:0/3628361078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0500041d0 con 0x555d795854d0 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.352+0000 7fb046ffd640 1 -- 172.21.15.19:0/3628361078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0500116a0 con 0x555d795854d0 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 -- 172.21.15.19:0/3628361078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 msgr2=0x555d7994efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb06145a640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb04c0afc60 0x7fb04c0b2170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb054005cc0 tx=0x7fb0540095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d7994efa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb05000bcb0 tx=0x7fb0500107a0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 -- 172.21.15.19:0/3628361078 shutdown_connections 2024-04-25T11:02:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d7995b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d7994efa0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 --2- 172.21.15.19:0/3628361078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7994ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 -- 172.21.15.19:0/3628361078 >> 172.21.15.19:0/3628361078 conn(0x555d795a24e0 msgr2=0x555d7994b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 -- 172.21.15.19:0/3628361078 shutdown_connections 2024-04-25T11:02:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 -- 172.21.15.19:0/3628361078 wait complete. 2024-04-25T11:02:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 -- start start 2024-04-25T11:02:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.353+0000 7fb069d73740 1 Processor -- start 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb069d73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79959660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb069d73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79959ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb069d73740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7995f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb069d73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d7995bfe0 con 0x555d795854d0 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb069d73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d7983bd10 con 0x555d7958d6b0 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb069d73740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555d7995c160 con 0x555d795ae390 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb069d73740 10 client.0 _ll_register_callbacks cb 0x555d79868210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79959660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79959660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45832/0 (socket says 172.21.15.19:45832) 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 -- 172.21.15.19:0/3969572322 learned_addr learned my addr 172.21.15.19:0/3969572322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb05bfff640 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79959ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb06145a640 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7995f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 -- 172.21.15.19:0/3969572322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 msgr2=0x555d7995f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7995f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 -- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 msgr2=0x555d79959ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79959ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.354+0000 7fb060c59640 1 -- 172.21.15.19:0/3969572322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d796907d0 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb060c59640 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79959660 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb050011010 tx=0x7fb05000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27458c09f0c44479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb03bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb03bfff640 1 -- 172.21.15.19:0/3969572322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05002a060 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb03bfff640 1 -- 172.21.15.19:0/3969572322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fb0500035f0 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0500041d0 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb03bfff640 1 -- 172.21.15.19:0/3969572322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb0500037f0 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 10 client.16728 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555d795c63f0 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb03bfff640 1 -- 172.21.15.19:0/3969572322 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0500041d0 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb03bfff640 1 -- 172.21.15.19:0/3969572322 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb050004880 con 0x555d795854d0 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 10 client.16728 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 20 client.16728 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d7956cf60 0x555d7996b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 4 client.16728 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.355+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d798c2990 con 0x555d7956cf60 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.356+0000 7fb05bfff640 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d7956cf60 0x555d7996b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.356+0000 7fb03bfff640 10 client.16728 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.356+0000 7fb05bfff640 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d7956cf60 0x555d7996b670 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879db3686666ac5c server_cookie=a1d5595e7469a25c in_seq=0 out_seq=0 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb03bfff640 1 -- 172.21.15.19:0/3969572322 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d798c2990 con 0x555d7956cf60 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb03bfff640 10 client.16728 handle_command_reply: tid=0 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 client.16728 shutdown 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 msgr2=0x555d79959660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79959660 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb050011010 tx=0x7fb05000bfd0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 shutdown_connections 2024-04-25T11:02:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d7956cf60 0x555d7996b670 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d7995f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79959ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 --2- 172.21.15.19:0/3969572322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79959660 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb03bfff640 0 client.16728 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 >> 172.21.15.19:0/3969572322 conn(0x555d795a24e0 msgr2=0x555d79950780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 shutdown_connections 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.357+0000 7fb069d73740 1 -- 172.21.15.19:0/3969572322 wait complete. 2024-04-25T11:02:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.358+0000 7fb069d73740 20 client.16728 trim_cache size 0 max 16384 2024-04-25T11:02:05.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:05 smithi019 ceph-mon[29209]: pgmap v232: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 Processor -- start 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 -- start start 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79958390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d7958d6b0 0x555d79950d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d795ae390 0x555d7995f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1ada70 con 0x555d795854d0 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1b02d0 con 0x555d795ae390 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb069d73740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1ae0c0 con 0x555d7958d6b0 2024-04-25T11:02:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79958390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79958390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45846/0 (socket says 172.21.15.19:45846) 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 -- 172.21.15.19:0/2211130010 learned_addr learned my addr 172.21.15.19:0/2211130010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb06145a640 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d795ae390 0x555d7995f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 -- 172.21.15.19:0/2211130010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d7958d6b0 msgr2=0x555d79950d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d7958d6b0 0x555d79950d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 -- 172.21.15.19:0/2211130010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d795ae390 msgr2=0x555d7995f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d795ae390 0x555d7995f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 -- 172.21.15.19:0/2211130010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d795860f0 con 0x555d795854d0 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb06145a640 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d795ae390 0x555d7995f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.376+0000 7fb060c59640 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79958390 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb050005d00 tx=0x7fb050000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edc45eda375621b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb0467fc640 1 -- 172.21.15.19:0/2211130010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05002a040 con 0x555d795854d0 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb0467fc640 1 -- 172.21.15.19:0/2211130010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0500035f0 con 0x555d795854d0 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb0467fc640 1 -- 172.21.15.19:0/2211130010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050011710 con 0x555d795854d0 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 -- 172.21.15.19:0/2211130010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 msgr2=0x555d79958390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79958390 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb050005d00 tx=0x7fb050000bc0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 -- 172.21.15.19:0/2211130010 shutdown_connections 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d795ae390 0x555d7995f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d7958d6b0 0x555d79950d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 --2- 172.21.15.19:0/2211130010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79958390 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 -- 172.21.15.19:0/2211130010 >> 172.21.15.19:0/2211130010 conn(0x555d795a24e0 msgr2=0x555d79950660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 -- 172.21.15.19:0/2211130010 shutdown_connections 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.377+0000 7fb069d73740 1 -- 172.21.15.19:0/2211130010 wait complete. 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 -- start start 2024-04-25T11:02:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 Processor -- start 2024-04-25T11:02:05.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79962040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79962580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d79962ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1ae790 con 0x555d795854d0 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1afc00 con 0x555d7958d6b0 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1aee60 con 0x555d795ae390 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb069d73740 10 client.0 _ll_register_callbacks cb 0x7fb0480a3c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79962040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb060c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79962040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45860/0 (socket says 172.21.15.19:45860) 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb060c59640 1 -- 172.21.15.19:0/616493624 learned_addr learned my addr 172.21.15.19:0/616493624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.378+0000 7fb06145a640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d79962ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb05bfff640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79962580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb060c59640 1 -- 172.21.15.19:0/616493624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 msgr2=0x555d79962ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb060c59640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d79962ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb060c59640 1 -- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 msgr2=0x555d79962580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb060c59640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79962580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb060c59640 1 -- 172.21.15.19:0/616493624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d798754b0 con 0x555d795854d0 2024-04-25T11:02:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb05bfff640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79962580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb060c59640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79962040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb050004d90 tx=0x7fb050003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effc91ab9d90a660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb03bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb03bfff640 1 -- 172.21.15.19:0/616493624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05003b020 con 0x555d795854d0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb03bfff640 1 -- 172.21.15.19:0/616493624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8513+0+0 (secure 0 0 0) 0x7fb050004080 con 0x555d795854d0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0500035f0 con 0x555d795854d0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb03bfff640 1 -- 172.21.15.19:0/616493624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb050046050 con 0x555d795854d0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb069d73740 10 client.16734 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555d799d29f0 con 0x555d795854d0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.379+0000 7fb03bfff640 1 -- 172.21.15.19:0/616493624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0500035f0 con 0x555d795854d0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb03bfff640 1 -- 172.21.15.19:0/616493624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb050011bf0 con 0x555d795854d0 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb069d73740 10 client.16734 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb069d73740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb069d73740 20 client.16734 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb069d73740 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d795423a0 0x555d799d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb069d73740 4 client.16734 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:02:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d798bc800 con 0x555d795423a0 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb05bfff640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d795423a0 0x555d799d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb03bfff640 10 client.16734 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb05bfff640 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d795423a0 0x555d799d5110 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6129304a29c94c0 server_cookie=a53cd8a6667648aa in_seq=0 out_seq=0 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb03bfff640 1 -- 172.21.15.19:0/616493624 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555d798bc800 con 0x555d795423a0 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb03bfff640 10 client.16734 handle_command_reply: tid=0 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.380+0000 7fb069d73740 1 client.16734 shutdown 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 msgr2=0x555d79962040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79962040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb050004d90 tx=0x7fb050003cc0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 shutdown_connections 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555d795423a0 0x555d799d5110 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d795ae390 0x555d79962ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb03bfff640 0 client.16734 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d7958d6b0 0x555d79962580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 --2- 172.21.15.19:0/616493624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d795854d0 0x555d79962040 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 >> 172.21.15.19:0/616493624 conn(0x555d795a24e0 msgr2=0x555d7995b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 shutdown_connections 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 1 -- 172.21.15.19:0/616493624 wait complete. 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb069d73740 20 client.16734 trim_cache size 0 max 16384 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "28de13c3-e3eb-4a67-b01f-bc9826a15d29": { 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "28de13c3-e3eb-4a67-b01f-bc9826a15d29", 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:02:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:02:05.388 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:02:05.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb04c0afc60 msgr2=0x7fb04c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb04c0afc60 0x7fb04c0b2170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb054005cc0 tx=0x7fb0540095c0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 msgr2=0x7fb05c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 0x7fb05c19ac60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb04800e9f0 tx=0x7fb04800eeb0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 shutdown_connections 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb04c0afc60 0x7fb04c0b2170 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb05c109fb0 0x7fb05c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.381+0000 7fb062ebd640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb05c109b10 0x7fb05c19ac60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.382+0000 7fb062ebd640 1 --2- 172.21.15.19:0/52418951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb05c104260 0x7fb05c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.382+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 >> 172.21.15.19:0/52418951 conn(0x7fb05c0ff230 msgr2=0x7fb05c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.382+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 shutdown_connections 2024-04-25T11:02:05.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:05.382+0000 7fb062ebd640 1 -- 172.21.15.19:0/52418951 wait complete. 2024-04-25T11:02:05.393 INFO:tasks.cephfs.filesystem:scrub status for tag:28de13c3-e3eb-4a67-b01f-bc9826a15d29 - {'path': '/', 'tag': '28de13c3-e3eb-4a67-b01f-bc9826a15d29', 'options': 'recursive,force'} 2024-04-25T11:02:05.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:05 smithi169 ceph-mon[33049]: pgmap v232: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T11:02:05.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:05 smithi112 ceph-mon[32799]: pgmap v232: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T11:02:06.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:06 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:02:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:06 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:06 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:02:06.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:06 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:06.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:06 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:02:06.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:06 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:06.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:06 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:02:06.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:06 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:06.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:06 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:02:06.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:06 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:06.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:06 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:02:06.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:06 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:07.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:07 smithi112 ceph-mon[32799]: pgmap v233: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-04-25T11:02:07.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:07 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:02:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:07 smithi019 ceph-mon[29209]: pgmap v233: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-04-25T11:02:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:07 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:02:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:07 smithi169 ceph-mon[33049]: pgmap v233: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-04-25T11:02:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:07 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:02:08.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:08 smithi112 ceph-mon[32799]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:02:08.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:08 smithi019 ceph-mon[29209]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:02:08.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:08 smithi169 ceph-mon[33049]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:02:09.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:09 smithi112 ceph-mon[32799]: pgmap v235: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:02:09.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:09 smithi019 ceph-mon[29209]: pgmap v235: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:02:09.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:09 smithi169 ceph-mon[33049]: pgmap v235: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:02:11.553 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:11 smithi169 ceph-mon[33049]: pgmap v236: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:02:11.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:11 smithi112 ceph-mon[32799]: pgmap v236: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:02:11.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:11 smithi019 ceph-mon[29209]: pgmap v236: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 53 KiB/s wr, 11 op/s 2024-04-25T11:02:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:13 smithi112 ceph-mon[32799]: pgmap v237: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:02:13.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:13 smithi112 ceph-mon[32799]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:02:13.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:13 smithi019 ceph-mon[29209]: pgmap v237: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:02:13.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:13 smithi019 ceph-mon[29209]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:02:13.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:13 smithi169 ceph-mon[33049]: pgmap v237: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:02:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:13 smithi169 ceph-mon[33049]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:02:15.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:15 smithi112 ceph-mon[32799]: pgmap v239: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:02:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:15 smithi019 ceph-mon[29209]: pgmap v239: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:02:15.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:15 smithi169 ceph-mon[33049]: pgmap v239: 97 pgs: 97 active+clean; 11 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:02:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:17 smithi112 ceph-mon[32799]: pgmap v240: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-25T11:02:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:17 smithi019 ceph-mon[29209]: pgmap v240: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-25T11:02:17.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:17 smithi169 ceph-mon[33049]: pgmap v240: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-25T11:02:19.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:19 smithi112 ceph-mon[32799]: pgmap v241: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:02:19.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:19 smithi019 ceph-mon[29209]: pgmap v241: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:02:19.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:19 smithi169 ceph-mon[33049]: pgmap v241: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:02:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:21 smithi112 ceph-mon[32799]: pgmap v242: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:02:21.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:21 smithi019 ceph-mon[29209]: pgmap v242: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:02:21.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:21 smithi169 ceph-mon[33049]: pgmap v242: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T11:02:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:23 smithi112 ceph-mon[32799]: pgmap v243: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:02:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:23 smithi019 ceph-mon[29209]: pgmap v243: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:02:23.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:23 smithi169 ceph-mon[33049]: pgmap v243: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:02:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:25 smithi112 ceph-mon[32799]: pgmap v244: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:02:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:25 smithi019 ceph-mon[29209]: pgmap v244: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:02:25.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:25 smithi169 ceph-mon[33049]: pgmap v244: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:02:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:27 smithi112 ceph-mon[32799]: pgmap v245: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:02:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:27 smithi019 ceph-mon[29209]: pgmap v245: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:02:27.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:27 smithi169 ceph-mon[33049]: pgmap v245: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:02:29.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:29 smithi112 ceph-mon[32799]: pgmap v246: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:29.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:29 smithi019 ceph-mon[29209]: pgmap v246: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:29.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:29 smithi169 ceph-mon[33049]: pgmap v246: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:31.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:31 smithi112 ceph-mon[32799]: pgmap v247: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:31.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:31 smithi019 ceph-mon[29209]: pgmap v247: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:31.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:31 smithi169 ceph-mon[33049]: pgmap v247: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:33.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:33 smithi112 ceph-mon[32799]: pgmap v248: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:33.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:33 smithi019 ceph-mon[29209]: pgmap v248: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:33.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:33 smithi169 ceph-mon[33049]: pgmap v248: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:35.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:02:35.394 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:02:35.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 -- 172.21.15.19:0/3646851195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 msgr2=0x7f1c881036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3646851195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c881036c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009900 tx=0x7f1c7802a410 comp rx=0 tx=0).stop 2024-04-25T11:02:35.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 -- 172.21.15.19:0/3646851195 shutdown_connections 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3646851195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c88104270 0x7f1c88110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3646851195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c88109fc0 0x7f1c88103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3646851195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c881036c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 -- 172.21.15.19:0/3646851195 >> 172.21.15.19:0/3646851195 conn(0x7f1c880ff1e0 msgr2=0x7f1c88101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 -- 172.21.15.19:0/3646851195 shutdown_connections 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.520+0000 7f1c8e492640 1 -- 172.21.15.19:0/3646851195 wait complete. 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.521+0000 7f1c8e492640 1 Processor -- start 2024-04-25T11:02:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.521+0000 7f1c8e492640 1 -- start start 2024-04-25T11:02:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c88104270 0x7f1c8819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c8819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c88109fc0 0x7f1c8819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88113d10 con 0x7f1c88109b20 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88113b90 con 0x7f1c88104270 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88113e90 con 0x7f1c88109fc0 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c8819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c8819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58492/0 (socket says 172.21.15.19:58492) 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 -- 172.21.15.19:0/3616464194 learned_addr learned my addr 172.21.15.19:0/3616464194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c87fff640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c88104270 0x7f1c8819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 -- 172.21.15.19:0/3616464194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c88109fc0 msgr2=0x7f1c8819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c88109fc0 0x7f1c8819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 -- 172.21.15.19:0/3616464194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c88104270 msgr2=0x7f1c8819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c88104270 0x7f1c8819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 -- 172.21.15.19:0/3616464194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8819f5b0 con 0x7f1c88109b20 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c87fff640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c88104270 0x7f1c8819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c877fe640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c8819acb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00d740 tx=0x7f1c7c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c857fa640 1 -- 172.21.15.19:0/3616464194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c017070 con 0x7f1c88109b20 2024-04-25T11:02:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8819f8a0 con 0x7f1c88109b20 2024-04-25T11:02:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c857fa640 1 -- 172.21.15.19:0/3616464194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c00de50 con 0x7f1c88109b20 2024-04-25T11:02:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c857fa640 1 -- 172.21.15.19:0/3616464194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c003c30 con 0x7f1c88109b20 2024-04-25T11:02:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.522+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c881a7220 con 0x7f1c88109b20 2024-04-25T11:02:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.525+0000 7f1c857fa640 1 -- 172.21.15.19:0/3616464194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1c7c003dd0 con 0x7f1c88109b20 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.525+0000 7f1c9541b740 1 Processor -- start 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.525+0000 7f1c9541b740 1 -- start start 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.525+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558501617390 0x5585019b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.525+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015f66b0 0x5585019c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c9541b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55850191dd80 con 0x5585015f66b0 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c9541b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585018d78b0 con 0x558501617390 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c9541b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558501919d40 con 0x5585015ee4d0 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015f66b0 0x5585019c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015f66b0 0x5585019c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58500/0 (socket says 172.21.15.19:58500) 2024-04-25T11:02:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 -- 172.21.15.19:0/1966673395 learned_addr learned my addr 172.21.15.19:0/1966673395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c8ca2f640 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558501617390 0x5585019b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c87fff640 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c857fa640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1c700afc60 0x7f1c700b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 -- 172.21.15.19:0/1966673395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 msgr2=0x5585019b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 -- 172.21.15.19:0/1966673395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558501617390 msgr2=0x5585019b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558501617390 0x5585019b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 -- 172.21.15.19:0/1966673395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585018a8a40 con 0x5585015f66b0 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c8ca2f640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1c700afc60 0x7f1c700b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c857fa640 1 -- 172.21.15.19:0/3616464194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f1c7c0deaa0 con 0x7f1c88109b20 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.526+0000 7f1c877fe640 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015f66b0 0x5585019c48f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0dc160 tx=0x7f1c7c0b82f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155c25ad53eb0437 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.527+0000 7f1c6e7fc640 1 -- 172.21.15.19:0/1966673395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0e4070 con 0x5585015f66b0 2024-04-25T11:02:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.527+0000 7f1c8ca2f640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1c700afc60 0x7f1c700b2170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c80007750 tx=0x7f1c800076e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.527+0000 7f1c6e7fc640 1 -- 172.21.15.19:0/1966673395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c0a6750 con 0x5585015f66b0 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.527+0000 7f1c6e7fc640 1 -- 172.21.15.19:0/1966673395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c020070 con 0x5585015f66b0 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.527+0000 7f1c9541b740 1 -- 172.21.15.19:0/1966673395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015f66b0 msgr2=0x5585019c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015f66b0 0x5585019c48f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0dc160 tx=0x7f1c7c0b82f0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 -- 172.21.15.19:0/1966673395 shutdown_connections 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015f66b0 0x5585019c48f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 --2- 172.21.15.19:0/1966673395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558501617390 0x5585019b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 -- 172.21.15.19:0/1966673395 >> 172.21.15.19:0/1966673395 conn(0x55850160b4e0 msgr2=0x5585019b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 -- 172.21.15.19:0/1966673395 shutdown_connections 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 -- 172.21.15.19:0/1966673395 wait complete. 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 -- start start 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.528+0000 7f1c9541b740 1 Processor -- start 2024-04-25T11:02:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015ee4d0 0x5585019c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558501617390 0x5585019c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585019c4f80 con 0x5585015ee4d0 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585018a4d10 con 0x5585015f66b0 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5585019c5100 con 0x558501617390 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 10 client.0 _ll_register_callbacks cb 0x5585018d1210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015ee4d0 0x5585019c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015ee4d0 0x5585019c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58514/0 (socket says 172.21.15.19:58514) 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 -- 172.21.15.19:0/4181784207 learned_addr learned my addr 172.21.15.19:0/4181784207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c877fe640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c8ca2f640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558501617390 0x5585019c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 -- 172.21.15.19:0/4181784207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558501617390 msgr2=0x5585019c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558501617390 0x5585019c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 -- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 msgr2=0x5585019c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 -- 172.21.15.19:0/4181784207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585016f97d0 con 0x5585015ee4d0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c877fe640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c8ca2f640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558501617390 0x5585019c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c87fff640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015ee4d0 0x5585019c2620 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005ab0 tx=0x7f1c78031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac94a61df405584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c677fe640 1 -- 172.21.15.19:0/4181784207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78035040 con 0x5585015ee4d0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c677fe640 1 -- 172.21.15.19:0/4181784207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f1c78004040 con 0x5585015ee4d0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.529+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c0a6750 con 0x5585015ee4d0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c677fe640 1 -- 172.21.15.19:0/4181784207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1c78004240 con 0x5585015ee4d0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 10 client.16746 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55850162f3f0 con 0x5585015ee4d0 2024-04-25T11:02:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c677fe640 1 -- 172.21.15.19:0/4181784207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c0a6750 con 0x5585015ee4d0 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c677fe640 1 -- 172.21.15.19:0/4181784207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1c78004a20 con 0x5585015ee4d0 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 10 client.16746 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 20 client.16746 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5585015d5f60 0x5585019d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 4 client.16746 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55850192b990 con 0x5585015d5f60 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.530+0000 7f1c877fe640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5585015d5f60 0x5585019d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.531+0000 7f1c877fe640 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5585015d5f60 0x5585019d46a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f29b52499e2e97 server_cookie=d0869eb3aa2320c5 in_seq=0 out_seq=0 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.531+0000 7f1c677fe640 10 client.16746 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c677fe640 1 -- 172.21.15.19:0/4181784207 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55850192b990 con 0x5585015d5f60 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c677fe640 10 client.16746 handle_command_reply: tid=0 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 client.16746 shutdown 2024-04-25T11:02:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015ee4d0 msgr2=0x5585019c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015ee4d0 0x5585019c2620 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005ab0 tx=0x7f1c78031890 comp rx=0 tx=0).stop 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 shutdown_connections 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5585015d5f60 0x5585019d46a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558501617390 0x5585019c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c677fe640 0 client.16746 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4181784207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015ee4d0 0x5585019c2620 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 >> 172.21.15.19:0/4181784207 conn(0x55850160b4e0 msgr2=0x5585019b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 shutdown_connections 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 1 -- 172.21.15.19:0/4181784207 wait complete. 2024-04-25T11:02:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.532+0000 7f1c9541b740 20 client.16746 trim_cache size 0 max 16384 2024-04-25T11:02:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.550+0000 7f1c9541b740 1 Processor -- start 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.550+0000 7f1c9541b740 1 -- start start 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015ee4d0 0x5585019b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015f66b0 0x5585019c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c9541b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ada10 con 0x5585015d5f60 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c9541b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b0270 con 0x5585015ee4d0 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c9541b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ae060 con 0x5585015f66b0 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58518/0 (socket says 172.21.15.19:58518) 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 -- 172.21.15.19:0/124372032 learned_addr learned my addr 172.21.15.19:0/124372032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c877fe640 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015ee4d0 0x5585019b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 -- 172.21.15.19:0/124372032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015f66b0 msgr2=0x5585019c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015f66b0 0x5585019c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 -- 172.21.15.19:0/124372032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015ee4d0 msgr2=0x5585019b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015ee4d0 0x5585019b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 -- 172.21.15.19:0/124372032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585015ef0f0 con 0x5585015d5f60 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c877fe640 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015ee4d0 0x5585019b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c87fff640 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c0f10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009850 tx=0x7f1c7802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f016a980fffefe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c6dffb640 1 -- 172.21.15.19:0/124372032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78035040 con 0x5585015d5f60 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c6dffb640 1 -- 172.21.15.19:0/124372032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c78040070 con 0x5585015d5f60 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.551+0000 7f1c6dffb640 1 -- 172.21.15.19:0/124372032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78004fc0 con 0x5585015d5f60 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 -- 172.21.15.19:0/124372032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 msgr2=0x5585019c0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c0f10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009850 tx=0x7f1c7802a410 comp rx=0 tx=0).stop 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 -- 172.21.15.19:0/124372032 shutdown_connections 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015f66b0 0x5585019c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015ee4d0 0x5585019b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 --2- 172.21.15.19:0/124372032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c0f10 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 -- 172.21.15.19:0/124372032 >> 172.21.15.19:0/124372032 conn(0x55850160b4e0 msgr2=0x5585019c39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 -- 172.21.15.19:0/124372032 shutdown_connections 2024-04-25T11:02:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 -- 172.21.15.19:0/124372032 wait complete. 2024-04-25T11:02:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.552+0000 7f1c9541b740 1 -- start start 2024-04-25T11:02:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 1 Processor -- start 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ae730 con 0x5585015d5f60 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881afba0 con 0x5585015f66b0 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881aee00 con 0x5585015ee4d0 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c9541b740 10 client.0 _ll_register_callbacks cb 0x7f1c780314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58532/0 (socket says 172.21.15.19:58532) 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c87fff640 1 -- 172.21.15.19:0/4083817643 learned_addr learned my addr 172.21.15.19:0/4083817643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c8ca2f640 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c877fe640 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c87fff640 1 -- 172.21.15.19:0/4083817643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 msgr2=0x5585019c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c87fff640 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c87fff640 1 -- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 msgr2=0x5585019ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.553+0000 7f1c87fff640 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c87fff640 1 -- 172.21.15.19:0/4083817643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585018de4b0 con 0x5585015d5f60 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c87fff640 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c9670 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009900 tx=0x7f1c780038e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbfdc38e72a997c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c677fe640 1 -- 172.21.15.19:0/4083817643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78035070 con 0x5585015d5f60 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c677fe640 1 -- 172.21.15.19:0/4083817643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f1c78031890 con 0x5585015d5f60 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c78040070 con 0x5585015d5f60 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c677fe640 1 -- 172.21.15.19:0/4083817643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1c78068050 con 0x5585015d5f60 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c9541b740 10 client.16752 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558501a3ba80 con 0x5585015d5f60 2024-04-25T11:02:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c677fe640 1 -- 172.21.15.19:0/4083817643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c78040070 con 0x5585015d5f60 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c677fe640 1 -- 172.21.15.19:0/4083817643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1c780051f0 con 0x5585015d5f60 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c9541b740 10 client.16752 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c9541b740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c9541b740 20 client.16752 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.554+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558501617390 0x558501a3e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c9541b740 4 client.16752 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558501925800 con 0x558501617390 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c877fe640 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558501617390 0x558501a3e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c877fe640 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558501617390 0x558501a3e100 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d4e8fb8cc7780d server_cookie=b8343177f4697ed3 in_seq=0 out_seq=0 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c677fe640 10 client.16752 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c677fe640 1 -- 172.21.15.19:0/4083817643 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558501925800 con 0x558501617390 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c677fe640 10 client.16752 handle_command_reply: tid=0 2024-04-25T11:02:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.555+0000 7f1c9541b740 1 client.16752 shutdown 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 msgr2=0x5585019c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c9670 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009900 tx=0x7f1c780038e0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 shutdown_connections 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x558501617390 0x558501a3e100 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585015f66b0 0x5585019ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585015ee4d0 0x5585019c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c677fe640 0 client.16752 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 --2- 172.21.15.19:0/4083817643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585015d5f60 0x5585019c9670 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 >> 172.21.15.19:0/4083817643 conn(0x55850160b4e0 msgr2=0x5585019c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 shutdown_connections 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 1 -- 172.21.15.19:0/4083817643 wait complete. 2024-04-25T11:02:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c9541b740 20 client.16752 trim_cache size 0 max 16384 2024-04-25T11:02:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:02:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:02:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:02:35.564 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1c700afc60 msgr2=0x7f1c700b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1c700afc60 0x7f1c700b2170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c80007750 tx=0x7f1c800076e0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 msgr2=0x7f1c8819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c8819acb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00d740 tx=0x7f1c7c00dc00 comp rx=0 tx=0).stop 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 shutdown_connections 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1c700afc60 0x7f1c700b2170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c88109fc0 0x7f1c8819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c88109b20 0x7f1c8819acb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.556+0000 7f1c8e492640 1 --2- 172.21.15.19:0/3616464194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c88104270 0x7f1c8819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.557+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 >> 172.21.15.19:0/3616464194 conn(0x7f1c880ff1e0 msgr2=0x7f1c880ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.557+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 shutdown_connections 2024-04-25T11:02:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.557+0000 7f1c8e492640 1 -- 172.21.15.19:0/3616464194 wait complete. 2024-04-25T11:02:35.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:35 smithi019 ceph-mon[29209]: pgmap v249: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:35.569 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:02:35.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 -- 172.21.15.19:0/519120251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109b20 msgr2=0x7fc8a4101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 --2- 172.21.15.19:0/519120251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109b20 0x7fc8a4101c10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009900 tx=0x7fc88c02a410 comp rx=0 tx=0).stop 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 -- 172.21.15.19:0/519120251 shutdown_connections 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 --2- 172.21.15.19:0/519120251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a41027c0 0x7fc8a4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 --2- 172.21.15.19:0/519120251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a4109fc0 0x7fc8a4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 --2- 172.21.15.19:0/519120251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109b20 0x7fc8a4101c10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 -- 172.21.15.19:0/519120251 >> 172.21.15.19:0/519120251 conn(0x7fc8a40fd860 msgr2=0x7fc8a40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 -- 172.21.15.19:0/519120251 shutdown_connections 2024-04-25T11:02:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.680+0000 7fc8ab030640 1 -- 172.21.15.19:0/519120251 wait complete. 2024-04-25T11:02:35.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 Processor -- start 2024-04-25T11:02:35.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 -- start start 2024-04-25T11:02:35.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a41027c0 0x7fc8a419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4109b20 0x7fc8a419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109fc0 0x7fc8a419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4113d10 con 0x7fc8a4109fc0 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4113b90 con 0x7fc8a41027c0 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8ab030640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4113e90 con 0x7fc8a4109b20 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109fc0 0x7fc8a419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109fc0 0x7fc8a419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58550/0 (socket says 172.21.15.19:58550) 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 -- 172.21.15.19:0/2107088598 learned_addr learned my addr 172.21.15.19:0/2107088598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a3fff640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4109b20 0x7fc8a419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a8dcc640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a41027c0 0x7fc8a419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 -- 172.21.15.19:0/2107088598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4109b20 msgr2=0x7fc8a419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4109b20 0x7fc8a419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 -- 172.21.15.19:0/2107088598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a41027c0 msgr2=0x7fc8a419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a41027c0 0x7fc8a419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.681+0000 7fc8a95cd640 1 -- 172.21.15.19:0/2107088598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419f550 con 0x7fc8a4109fc0 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.682+0000 7fc8a95cd640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109fc0 0x7fc8a419ef10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00eaa0 tx=0x7fc89c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.682+0000 7fc8a1ffb640 1 -- 172.21.15.19:0/2107088598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00cdb0 con 0x7fc8a4109fc0 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.682+0000 7fc8a1ffb640 1 -- 172.21.15.19:0/2107088598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c005020 con 0x7fc8a4109fc0 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.682+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a419f840 con 0x7fc8a4109fc0 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.682+0000 7fc8a1ffb640 1 -- 172.21.15.19:0/2107088598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0116d0 con 0x7fc8a4109fc0 2024-04-25T11:02:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.682+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41a71c0 con 0x7fc8a4109fc0 2024-04-25T11:02:35.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.683+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a406a190 con 0x7fc8a4109fc0 2024-04-25T11:02:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.684+0000 7fc8a1ffb640 1 -- 172.21.15.19:0/2107088598 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc89c0051d0 con 0x7fc8a4109fc0 2024-04-25T11:02:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.685+0000 7fc8a1ffb640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc8900afce0 0x7fc8900b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.685+0000 7fc8a8dcc640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc8900afce0 0x7fc8900b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.685+0000 7fc8a1ffb640 1 -- 172.21.15.19:0/2107088598 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7fc89c0df270 con 0x7fc8a4109fc0 2024-04-25T11:02:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.686+0000 7fc8a8dcc640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc8900afce0 0x7fc8900b21f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc88c031440 tx=0x7fc88c005d50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.690+0000 7fc8a1ffb640 1 -- 172.21.15.19:0/2107088598 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc89c016030 con 0x7fc8a4109fc0 2024-04-25T11:02:35.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:35 smithi169 ceph-mon[33049]: pgmap v249: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:35.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:35 smithi112 ceph-mon[32799]: pgmap v249: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.915+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8a41a73c0 con 0x7fc8a4109fc0 2024-04-25T11:02:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.916+0000 7fc8a1ffb640 1 -- 172.21.15.19:0/2107088598 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fc89c0a7b10 con 0x7fc8a4109fc0 2024-04-25T11:02:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:02:35.917 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:02:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:02:35.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc8900afce0 msgr2=0x7fc8900b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc8900afce0 0x7fc8900b21f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc88c031440 tx=0x7fc88c005d50 comp rx=0 tx=0).stop 2024-04-25T11:02:35.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109fc0 msgr2=0x7fc8a419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109fc0 0x7fc8a419ef10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00eaa0 tx=0x7fc89c00ef60 comp rx=0 tx=0).stop 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 shutdown_connections 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc8900afce0 0x7fc8900b21f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a4109fc0 0x7fc8a419ef10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4109b20 0x7fc8a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 --2- 172.21.15.19:0/2107088598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a41027c0 0x7fc8a419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 >> 172.21.15.19:0/2107088598 conn(0x7fc8a40fd860 msgr2=0x7fc8a40ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 shutdown_connections 2024-04-25T11:02:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:35.918+0000 7fc8ab030640 1 -- 172.21.15.19:0/2107088598 wait complete. 2024-04-25T11:02:35.931 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 -- 172.21.15.19:0/839941994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c8108eb0 msgr2=0x7fe2c8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/839941994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c8108eb0 0x7fe2c8102b50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009900 tx=0x7fe2bc02a410 comp rx=0 tx=0).stop 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 -- 172.21.15.19:0/839941994 shutdown_connections 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/839941994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c81031c0 0x7fe2c810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/839941994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c8108eb0 0x7fe2c8102b50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/839941994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c8108a10 0x7fe2c8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 -- 172.21.15.19:0/839941994 >> 172.21.15.19:0/839941994 conn(0x7fe2c8078110 msgr2=0x7fe2c8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 -- 172.21.15.19:0/839941994 shutdown_connections 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.045+0000 7fe2cdbee640 1 -- 172.21.15.19:0/839941994 wait complete. 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 Processor -- start 2024-04-25T11:02:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 -- start start 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c81031c0 0x7fe2c8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c8108a10 0x7fe2c8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c8108eb0 0x7fe2c819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8112c30 con 0x7fe2c81031c0 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8112ab0 con 0x7fe2c8108a10 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2cdbee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8112db0 con 0x7fe2c8108eb0 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c81031c0 0x7fe2c8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.046+0000 7fe2c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c81031c0 0x7fe2c8199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58572/0 (socket says 172.21.15.19:58572) 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c77fe640 1 -- 172.21.15.19:0/3292619419 learned_addr learned my addr 172.21.15.19:0/3292619419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c8108eb0 0x7fe2c819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c8108a10 0x7fe2c8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c77fe640 1 -- 172.21.15.19:0/3292619419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c8108eb0 msgr2=0x7fe2c819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c8108eb0 0x7fe2c819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c77fe640 1 -- 172.21.15.19:0/3292619419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c8108a10 msgr2=0x7fe2c8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c8108a10 0x7fe2c8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c77fe640 1 -- 172.21.15.19:0/3292619419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c819e4d0 con 0x7fe2c81031c0 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c8108a10 0x7fe2c8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c81031c0 0x7fe2c8199670 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800e9f0 tx=0x7fe2b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c4ff9640 1 -- 172.21.15.19:0/3292619419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800ccc0 con 0x7fe2c81031c0 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c4ff9640 1 -- 172.21.15.19:0/3292619419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b80108e0 con 0x7fe2c81031c0 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c819e7c0 con 0x7fe2c81031c0 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2c4ff9640 1 -- 172.21.15.19:0/3292619419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8018860 con 0x7fe2c81031c0 2024-04-25T11:02:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.047+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c81a6140 con 0x7fe2c81031c0 2024-04-25T11:02:36.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 Processor -- start 2024-04-25T11:02:36.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 -- start start 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2c4ff9640 1 -- 172.21.15.19:0/3292619419 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe2b8010430 con 0x7fe2c81031c0 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac10b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd414d0 0x5632ac10bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac1186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5632ac070d80 con 0x5632abd414d0 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632ac02aa20 con 0x5632abd6a390 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5632ac06cd40 con 0x5632abd496b0 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd414d0 0x5632ac10bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd414d0 0x5632ac10bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58582/0 (socket says 172.21.15.19:58582) 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2c77fe640 1 -- 172.21.15.19:0/826169014 learned_addr learned my addr 172.21.15.19:0/826169014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac10b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac1186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.050+0000 7fe2c4ff9640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2b00afce0 0x7fe2b00b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c77fe640 1 -- 172.21.15.19:0/826169014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 msgr2=0x5632ac1186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac1186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c77fe640 1 -- 172.21.15.19:0/826169014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 msgr2=0x5632ac10b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac10b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c77fe640 1 -- 172.21.15.19:0/826169014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ac035c30 con 0x5632abd414d0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2b00afce0 0x7fe2b00b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c4ff9640 1 -- 172.21.15.19:0/3292619419 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7fe2b80df380 con 0x7fe2c81031c0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd414d0 0x5632ac10bd50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80dc9b0 tx=0x7fe2b806d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d5dfeb4b68f81f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2adffb640 1 -- 172.21.15.19:0/826169014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80a6e10 con 0x5632abd414d0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2adffb640 1 -- 172.21.15.19:0/826169014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b8010720 con 0x5632abd414d0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2adffb640 1 -- 172.21.15.19:0/826169014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80a6c30 con 0x5632abd414d0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.051+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2b00afce0 0x7fe2b00b21f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5632ac108980 tx=0x7fe2c0004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 -- 172.21.15.19:0/826169014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd414d0 msgr2=0x5632ac10bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd414d0 0x5632ac10bd50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80dc9b0 tx=0x7fe2b806d990 comp rx=0 tx=0).stop 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 -- 172.21.15.19:0/826169014 shutdown_connections 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac1186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd414d0 0x5632ac10bd50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/826169014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac10b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 -- 172.21.15.19:0/826169014 >> 172.21.15.19:0/826169014 conn(0x5632abd5e4e0 msgr2=0x5632ac107fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 -- 172.21.15.19:0/826169014 shutdown_connections 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 -- 172.21.15.19:0/826169014 wait complete. 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.052+0000 7fe2d4b75740 1 -- start start 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 1 Processor -- start 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac116250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac116790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac1221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5632ac118d30 con 0x5632abd496b0 2024-04-25T11:02:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632abff7dc0 con 0x5632abd6a390 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5632ac118eb0 con 0x5632abd414d0 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2d4b75740 10 client.0 _ll_register_callbacks cb 0x5632ac013600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac116790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac116790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58586/0 (socket says 172.21.15.19:58586) 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/1073338687 learned_addr learned my addr 172.21.15.19:0/1073338687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac1221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/1073338687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 msgr2=0x5632ac116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 msgr2=0x5632ac1221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac1221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/1073338687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632abd823f0 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.053+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac116790 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005ab0 tx=0x7fe2bc032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4a43ddf83297de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/1073338687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc040070 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/1073338687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7fe2bc004040 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632ac07e990 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/1073338687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe2bc035040 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2d4b75740 10 client.16770 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5632abe4c7d0 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/1073338687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5632ac07e990 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.054+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/1073338687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe2bc0048f0 con 0x5632abd496b0 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2d4b75740 10 client.16770 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2d4b75740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2d4b75740 20 client.16770 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abd28f60 0x5632ac128670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2d4b75740 4 client.16770 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632ac078800 con 0x5632abd28f60 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abd28f60 0x5632ac128670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2a6ffd640 10 client.16770 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.055+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abd28f60 0x5632ac128670 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593c4d39a6f462e6 server_cookie=e050ba9026d155da in_seq=0 out_seq=0 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/1073338687 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5632ac078800 con 0x5632abd28f60 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2a6ffd640 10 client.16770 handle_command_reply: tid=0 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 client.16770 shutdown 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 msgr2=0x5632ac116790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac116790 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005ab0 tx=0x7fe2bc032eb0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 shutdown_connections 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abd28f60 0x5632ac128670 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac1221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2a6ffd640 0 client.16770 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:36.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac116790 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/1073338687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 >> 172.21.15.19:0/1073338687 conn(0x5632abd5e4e0 msgr2=0x5632ac10d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 shutdown_connections 2024-04-25T11:02:36.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.056+0000 7fe2d4b75740 1 -- 172.21.15.19:0/1073338687 wait complete. 2024-04-25T11:02:36.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.057+0000 7fe2d4b75740 20 client.16770 trim_cache size 0 max 16384 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 Processor -- start 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 -- start start 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac115020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac121e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac11e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81ac9f0 con 0x5632abd496b0 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81af250 con 0x5632abd6a390 2024-04-25T11:02:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81ad040 con 0x5632abd414d0 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac121e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac121e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58602/0 (socket says 172.21.15.19:58602) 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/3042466371 learned_addr learned my addr 172.21.15.19:0/3042466371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac11e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/3042466371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 msgr2=0x5632ac115020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac115020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/3042466371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 msgr2=0x5632ac11e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac11e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.071+0000 7fe2c6ffd640 1 -- 172.21.15.19:0/3042466371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ac114590 con 0x5632abd496b0 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.072+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac121e10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc004d40 tx=0x7fe2bc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93714f339a4aa22b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.072+0000 7fe2ad7fa640 1 -- 172.21.15.19:0/3042466371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc051040 con 0x5632abd496b0 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.072+0000 7fe2ad7fa640 1 -- 172.21.15.19:0/3042466371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc031d20 con 0x5632abd496b0 2024-04-25T11:02:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.072+0000 7fe2ad7fa640 1 -- 172.21.15.19:0/3042466371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc040040 con 0x5632abd496b0 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.072+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3042466371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 msgr2=0x5632ac121e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.072+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac121e10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc004d40 tx=0x7fe2bc001e70 comp rx=0 tx=0).stop 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3042466371 shutdown_connections 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd6a390 0x5632ac11e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd496b0 0x5632ac121e10 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3042466371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd414d0 0x5632ac115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3042466371 >> 172.21.15.19:0/3042466371 conn(0x5632abd5e4e0 msgr2=0x5632ac114910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3042466371 shutdown_connections 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3042466371 wait complete. 2024-04-25T11:02:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- start start 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 Processor -- start 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd414d0 0x5632ac113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac12d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd6a390 0x5632ac12de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81ad710 con 0x5632abd6a390 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81aeb80 con 0x5632abd414d0 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81adde0 con 0x5632abd496b0 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.073+0000 7fe2d4b75740 10 client.0 _ll_register_callbacks cb 0x7fe2bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd6a390 0x5632ac12de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd6a390 0x5632ac12de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58608/0 (socket says 172.21.15.19:58608) 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 -- 172.21.15.19:0/3134777731 learned_addr learned my addr 172.21.15.19:0/3134777731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c6ffd640 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac12d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd414d0 0x5632ac113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 -- 172.21.15.19:0/3134777731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 msgr2=0x5632ac12d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac12d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 -- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd414d0 msgr2=0x5632ac113ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd414d0 0x5632ac113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 -- 172.21.15.19:0/3134777731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ac12e560 con 0x5632abd6a390 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2c7fff640 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd6a390 0x5632ac12de00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0010980 tx=0x7fe2c0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f02c6e242947af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/3134777731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c002d060 con 0x5632abd6a390 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/3134777731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7fe2c001bb30 con 0x5632abd6a390 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2bc031d20 con 0x5632abd6a390 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/3134777731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe2c001bd30 con 0x5632abd6a390 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2d4b75740 10 client.16776 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.074+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5632ac1929f0 con 0x5632abd6a390 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/3134777731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc031d20 con 0x5632abd6a390 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/3134777731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe2c000f040 con 0x5632abd6a390 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2d4b75740 10 client.16776 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2d4b75740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2d4b75740 20 client.16776 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abcfe3a0 0x5632ac1951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2d4b75740 4 client.16776 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632abff3610 con 0x5632abcfe3a0 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.075+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abcfe3a0 0x5632ac1951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2c77fe640 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abcfe3a0 0x5632ac1951e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21530e7f226dd331 server_cookie=bc2788f89e87d12b in_seq=0 out_seq=0 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2a6ffd640 10 client.16776 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2a6ffd640 1 -- 172.21.15.19:0/3134777731 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632abff3610 con 0x5632abcfe3a0 2024-04-25T11:02:36.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2a6ffd640 10 client.16776 handle_command_reply: tid=0 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 client.16776 shutdown 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd6a390 msgr2=0x5632ac12de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd6a390 0x5632ac12de00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0010980 tx=0x7fe2c0015040 comp rx=0 tx=0).stop 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 shutdown_connections 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x5632abcfe3a0 0x5632ac1951e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632abd6a390 0x5632ac12de00 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632abd496b0 0x5632ac12d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2a6ffd640 0 client.16776 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 --2- 172.21.15.19:0/3134777731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632abd414d0 0x5632ac113ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 >> 172.21.15.19:0/3134777731 conn(0x5632abd5e4e0 msgr2=0x5632ac106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 shutdown_connections 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 1 -- 172.21.15.19:0/3134777731 wait complete. 2024-04-25T11:02:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.076+0000 7fe2d4b75740 20 client.16776 trim_cache size 0 max 16384 2024-04-25T11:02:36.084 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:02:36.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2b00afce0 msgr2=0x7fe2b00b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2b00afce0 0x7fe2b00b21f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5632ac108980 tx=0x7fe2c0004a40 comp rx=0 tx=0).stop 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c81031c0 msgr2=0x7fe2c8199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c81031c0 0x7fe2c8199670 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800e9f0 tx=0x7fe2b800eeb0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 shutdown_connections 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe2b00afce0 0x7fe2b00b21f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c8108eb0 0x7fe2c819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c8108a10 0x7fe2c8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 --2- 172.21.15.19:0/3292619419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c81031c0 0x7fe2c8199670 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 >> 172.21.15.19:0/3292619419 conn(0x7fe2c8078110 msgr2=0x7fe2c81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 shutdown_connections 2024-04-25T11:02:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:36.077+0000 7fe2cdbee640 1 -- 172.21.15.19:0/3292619419 wait complete. 2024-04-25T11:02:36.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:36 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2107088598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:02:36.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:36 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2107088598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:02:36.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:36 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2107088598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:02:37.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:02:37.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:02:37.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:02:37.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 -- 172.21.15.19:0/3858298486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 msgr2=0x7f4fac102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/3858298486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac102020 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009900 tx=0x7f4f9402a410 comp rx=0 tx=0).stop 2024-04-25T11:02:37.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 -- 172.21.15.19:0/3858298486 shutdown_connections 2024-04-25T11:02:37.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/3858298486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac106e30 0x7f4fac107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/3858298486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fac1025f0 0x7f4fac1068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/3858298486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac102020 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 -- 172.21.15.19:0/3858298486 >> 172.21.15.19:0/3858298486 conn(0x7f4fac077880 msgr2=0x7f4fac077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 -- 172.21.15.19:0/3858298486 shutdown_connections 2024-04-25T11:02:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.220+0000 7f4fb2d96640 1 -- 172.21.15.19:0/3858298486 wait complete. 2024-04-25T11:02:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 Processor -- start 2024-04-25T11:02:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 -- start start 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac0feca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fac1025f0 0x7f4fac0ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac106e30 0x7f4fac0ff720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac112bc0 con 0x7f4fac06b100 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac112a40 con 0x7f4fac106e30 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb2d96640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac112d40 con 0x7f4fac1025f0 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb0b32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac0feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb0b32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac0feca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58626/0 (socket says 172.21.15.19:58626) 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.221+0000 7f4fb0b32640 1 -- 172.21.15.19:0/1275167892 learned_addr learned my addr 172.21.15.19:0/1275167892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fabfff640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fac1025f0 0x7f4fac0ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb1333640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac106e30 0x7f4fac0ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb0b32640 1 -- 172.21.15.19:0/1275167892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fac1025f0 msgr2=0x7f4fac0ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fac1025f0 0x7f4fac0ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb0b32640 1 -- 172.21.15.19:0/1275167892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac106e30 msgr2=0x7f4fac0ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac106e30 0x7f4fac0ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb0b32640 1 -- 172.21.15.19:0/1275167892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fac1a8a20 con 0x7f4fac06b100 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fabfff640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fac1025f0 0x7f4fac0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb1333640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac106e30 0x7f4fac0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac0feca0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4f94005940 tx=0x7f4f940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fa9ffb640 1 -- 172.21.15.19:0/1275167892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9403f030 con 0x7f4fac06b100 2024-04-25T11:02:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fa9ffb640 1 -- 172.21.15.19:0/1275167892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f94003cb0 con 0x7f4fac06b100 2024-04-25T11:02:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fac1a8cb0 con 0x7f4fac06b100 2024-04-25T11:02:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fa9ffb640 1 -- 172.21.15.19:0/1275167892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94035070 con 0x7f4fac06b100 2024-04-25T11:02:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.222+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fac1a91a0 con 0x7f4fac06b100 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 Processor -- start 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fa9ffb640 1 -- 172.21.15.19:0/1275167892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4f94042030 con 0x7f4fac06b100 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 -- start start 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f389620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f389b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f38a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55887f2efb00 con 0x55887efe9410 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55887f28b920 con 0x55887efc8730 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55887f2ebac0 con 0x55887efc0550 2024-04-25T11:02:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb1333640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f389620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb1333640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f389620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58632/0 (socket says 172.21.15.19:58632) 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.225+0000 7f4fb1333640 1 -- 172.21.15.19:0/2879889863 learned_addr learned my addr 172.21.15.19:0/2879889863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fabfff640 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f38a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f389b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fa9ffb640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4f980afc60 0x7f4f980b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb1333640 1 -- 172.21.15.19:0/2879889863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 msgr2=0x55887f389b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb1333640 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f389b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb1333640 1 -- 172.21.15.19:0/2879889863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 msgr2=0x55887f38a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb1333640 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f38a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb1333640 1 -- 172.21.15.19:0/2879889863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887f0c97e0 con 0x55887efe9410 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fabfff640 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f38a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fa9ffb640 1 -- 172.21.15.19:0/1275167892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f4f940ff9e0 con 0x7f4fac06b100 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb1333640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4f980afc60 0x7f4f980b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fb1333640 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f389620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009c90 tx=0x7f4fa400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952ba63ef4b2afc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fa2ffd640 1 -- 172.21.15.19:0/2879889863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa401a410 con 0x55887efe9410 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fa2ffd640 1 -- 172.21.15.19:0/2879889863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa40054e0 con 0x55887efe9410 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.226+0000 7f4fa2ffd640 1 -- 172.21.15.19:0/2879889863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4004c90 con 0x55887efe9410 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2879889863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 msgr2=0x55887f389620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f389620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009c90 tx=0x7f4fa400e6c0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb1333640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4f980afc60 0x7f4f980b2170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fa401f000 tx=0x7f4fa4020000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2879889863 shutdown_connections 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f38a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f389b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2879889863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f389620 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2879889863 >> 172.21.15.19:0/2879889863 conn(0x55887efdd560 msgr2=0x55887f385dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2879889863 shutdown_connections 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.227+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2879889863 wait complete. 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 -- start start 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 Processor -- start 2024-04-25T11:02:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f393c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f3941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f399df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55887f396970 con 0x55887efe9410 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55887f2895d0 con 0x55887efc8730 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55887f396af0 con 0x55887efc0550 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.228+0000 7f4fb9c47740 10 client.0 _ll_register_callbacks cb 0x55887f277080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f399df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f399df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58646/0 (socket says 172.21.15.19:58646) 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 -- 172.21.15.19:0/2758857571 learned_addr learned my addr 172.21.15.19:0/2758857571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fabfff640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f3941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f393c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 -- 172.21.15.19:0/2758857571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 msgr2=0x55887f393c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f393c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 -- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 msgr2=0x55887f3941b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f3941b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 -- 172.21.15.19:0/2758857571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887f129f70 con 0x55887efe9410 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f393c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fabfff640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f3941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb1333640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f399df0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4004850 tx=0x7f4fa4009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e70e55c92028d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4f8bfff640 1 -- 172.21.15.19:0/2758857571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400ee90 con 0x55887efe9410 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4f8bfff640 1 -- 172.21.15.19:0/2758857571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f4fa4021070 con 0x55887efe9410 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa40054e0 con 0x55887efe9410 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.229+0000 7f4f8bfff640 1 -- 172.21.15.19:0/2758857571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4fa4005690 con 0x55887efe9410 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 10 client.16788 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55887efc1170 con 0x55887efe9410 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4f8bfff640 1 -- 172.21.15.19:0/2758857571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa40054e0 con 0x55887efe9410 2024-04-25T11:02:37.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4f8bfff640 1 -- 172.21.15.19:0/2758857571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4fa400f040 con 0x55887efe9410 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 10 client.16788 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 20 client.16788 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887efa81f0 0x55887f3a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 4 client.16788 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55887f2fd710 con 0x55887efa81f0 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.230+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887efa81f0 0x55887f3a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.231+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887efa81f0 0x55887f3a5ff0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc11611581b782f0 server_cookie=2893bfccd5dbc388 in_seq=0 out_seq=0 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.231+0000 7f4f8bfff640 10 client.16788 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.231+0000 7f4f8bfff640 1 -- 172.21.15.19:0/2758857571 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55887f2fd710 con 0x55887efa81f0 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4f8bfff640 10 client.16788 handle_command_reply: tid=0 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 client.16788 shutdown 2024-04-25T11:02:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 msgr2=0x55887f399df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f399df0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4004850 tx=0x7f4fa4009df0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 shutdown_connections 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887efa81f0 0x55887f3a5ff0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efe9410 0x55887f399df0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f3941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4f8bfff640 0 client.16788 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/2758857571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efc0550 0x55887f393c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 >> 172.21.15.19:0/2758857571 conn(0x55887efdd560 msgr2=0x55887f389450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 shutdown_connections 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 1 -- 172.21.15.19:0/2758857571 wait complete. 2024-04-25T11:02:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.232+0000 7f4fb9c47740 20 client.16788 trim_cache size 0 max 16384 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.250+0000 7f4fb9c47740 1 Processor -- start 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb9c47740 1 -- start start 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc0550 0x55887f38d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f395bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f399d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1afb60 con 0x55887efc0550 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1b23c0 con 0x55887efc8730 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1b01b0 con 0x55887efe9410 2024-04-25T11:02:37.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc0550 0x55887f38d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc0550 0x55887f38d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58656/0 (socket says 172.21.15.19:58656) 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 -- 172.21.15.19:0/3872429421 learned_addr learned my addr 172.21.15.19:0/3872429421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fabfff640 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f395bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb1333640 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f399d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 -- 172.21.15.19:0/3872429421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 msgr2=0x55887f399d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f399d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 -- 172.21.15.19:0/3872429421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 msgr2=0x55887f395bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f395bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 -- 172.21.15.19:0/3872429421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887f38ab20 con 0x55887efc0550 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fabfff640 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f395bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb1333640 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f399d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc0550 0x55887f38d760 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4f940c7880 tx=0x7f4f940ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171303e7667ef0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.251+0000 7f4fa27fc640 1 -- 172.21.15.19:0/3872429421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f940a9c70 con 0x55887efc0550 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fa27fc640 1 -- 172.21.15.19:0/3872429421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f940d0320 con 0x55887efc0550 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fa27fc640 1 -- 172.21.15.19:0/3872429421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94049040 con 0x55887efc0550 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 -- 172.21.15.19:0/3872429421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc0550 msgr2=0x55887f38d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc0550 0x55887f38d760 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4f940c7880 tx=0x7f4f940ea040 comp rx=0 tx=0).stop 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 -- 172.21.15.19:0/3872429421 shutdown_connections 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f399d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc8730 0x55887f395bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/3872429421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc0550 0x55887f38d760 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 -- 172.21.15.19:0/3872429421 >> 172.21.15.19:0/3872429421 conn(0x55887efdd560 msgr2=0x55887f384b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 -- 172.21.15.19:0/3872429421 shutdown_connections 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.252+0000 7f4fb9c47740 1 -- 172.21.15.19:0/3872429421 wait complete. 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 -- start start 2024-04-25T11:02:37.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 Processor -- start 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc0550 0x55887f3a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc8730 0x55887f3a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f39d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1b0880 con 0x55887efc8730 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1b1cf0 con 0x55887efc0550 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1b0f50 con 0x55887efe9410 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fb9c47740 10 client.0 _ll_register_callbacks cb 0x7f4f940c41d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc8730 0x55887f3a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc8730 0x55887f3a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58670/0 (socket says 172.21.15.19:58670) 2024-04-25T11:02:37.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.253+0000 7f4fabfff640 1 -- 172.21.15.19:0/140477653 learned_addr learned my addr 172.21.15.19:0/140477653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fb1333640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f39d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc0550 0x55887f3a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fabfff640 1 -- 172.21.15.19:0/140477653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 msgr2=0x55887f39d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fabfff640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f39d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fabfff640 1 -- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc0550 msgr2=0x55887f3a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fabfff640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc0550 0x55887f3a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fabfff640 1 -- 172.21.15.19:0/140477653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887f39dd80 con 0x55887efc8730 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fb1333640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f39d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fabfff640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc8730 0x55887f3a0770 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c011b50 tx=0x7f4f9c00e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c5f24aea352730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4f8bfff640 1 -- 172.21.15.19:0/140477653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c01b420 con 0x55887efc8730 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4f8bfff640 1 -- 172.21.15.19:0/140477653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f4f9c00f420 con 0x55887efc8730 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55887f2f7580 con 0x55887efc8730 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4f8bfff640 1 -- 172.21.15.19:0/140477653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c00f620 con 0x55887efc8730 2024-04-25T11:02:37.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fb9c47740 10 client.16794 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55887f40da40 con 0x55887efc8730 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.254+0000 7f4f8bfff640 1 -- 172.21.15.19:0/140477653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55887f2f7580 con 0x55887efc8730 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4f8bfff640 1 -- 172.21.15.19:0/140477653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4f9c01b970 con 0x55887efc8730 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb9c47740 10 client.16794 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb9c47740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb9c47740 20 client.16794 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887ef7d630 0x55887f4102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb9c47740 4 client.16794 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55887f272ac0 con 0x55887ef7d630 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887ef7d630 0x55887f4102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4f8bfff640 10 client.16794 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.255+0000 7f4fb0b32640 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887ef7d630 0x55887f4102b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72058eeacdbf4e22 server_cookie=b4ab310dd3991e8f in_seq=0 out_seq=0 2024-04-25T11:02:37.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.256+0000 7f4f8bfff640 1 -- 172.21.15.19:0/140477653 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55887f272ac0 con 0x55887ef7d630 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.256+0000 7f4f8bfff640 10 client.16794 handle_command_reply: tid=0 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 client.16794 shutdown 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc8730 msgr2=0x55887f3a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc8730 0x55887f3a0770 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c011b50 tx=0x7f4f9c00e780 comp rx=0 tx=0).stop 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 shutdown_connections 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55887ef7d630 0x55887f4102b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55887efe9410 0x55887f39d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55887efc8730 0x55887f3a0770 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4f8bfff640 0 client.16794 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 --2- 172.21.15.19:0/140477653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55887efc0550 0x55887f3a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 >> 172.21.15.19:0/140477653 conn(0x55887efdd560 msgr2=0x55887f3893b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 shutdown_connections 2024-04-25T11:02:37.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 1 -- 172.21.15.19:0/140477653 wait complete. 2024-04-25T11:02:37.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.257+0000 7f4fb9c47740 20 client.16794 trim_cache size 0 max 16384 2024-04-25T11:02:37.264 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:02:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:02:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ca7c7633-6708-40da-b17d-e46bf31723c8", 2024-04-25T11:02:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:02:37.264 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:02:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4f980afc60 msgr2=0x7f4f980b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4f980afc60 0x7f4f980b2170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fa401f000 tx=0x7f4fa4020000 comp rx=0 tx=0).stop 2024-04-25T11:02:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 msgr2=0x7f4fac0feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac0feca0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4f94005940 tx=0x7f4f940042f0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 shutdown_connections 2024-04-25T11:02:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4f980afc60 0x7f4f980b2170 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac106e30 0x7f4fac0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fac1025f0 0x7f4fac0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 --2- 172.21.15.19:0/1275167892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fac06b100 0x7f4fac0feca0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 >> 172.21.15.19:0/1275167892 conn(0x7f4fac077880 msgr2=0x7f4fac108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 shutdown_connections 2024-04-25T11:02:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.258+0000 7f4fb2d96640 1 -- 172.21.15.19:0/1275167892 wait complete. 2024-04-25T11:02:37.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:02:37.270 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:02:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 -- 172.21.15.19:0/260707447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4674104bf0 msgr2=0x7f4674105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 --2- 172.21.15.19:0/260707447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4674104bf0 0x7f4674105000 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4664009900 tx=0x7f466402a410 comp rx=0 tx=0).stop 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 -- 172.21.15.19:0/260707447 shutdown_connections 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 --2- 172.21.15.19:0/260707447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f467410e040 0x7f4674110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 --2- 172.21.15.19:0/260707447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467410b650 0x7f467410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 --2- 172.21.15.19:0/260707447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4674104bf0 0x7f4674105000 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 -- 172.21.15.19:0/260707447 >> 172.21.15.19:0/260707447 conn(0x7f4674078100 msgr2=0x7f4674078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 -- 172.21.15.19:0/260707447 shutdown_connections 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 -- 172.21.15.19:0/260707447 wait complete. 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.384+0000 7f467a2b1640 1 Processor -- start 2024-04-25T11:02:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467a2b1640 1 -- start start 2024-04-25T11:02:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467a2b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674104bf0 0x7f46741024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467a2b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467410b650 0x7f46741029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467a2b1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467410e040 0x7f4674102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467a2b1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4674113d30 con 0x7f467410b650 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467a2b1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4674113bb0 con 0x7f4674104bf0 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467a2b1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4674113eb0 con 0x7f467410e040 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f46737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467410b650 0x7f46741029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f46737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467410b650 0x7f46741029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58694/0 (socket says 172.21.15.19:58694) 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f46737fe640 1 -- 172.21.15.19:0/3722607561 learned_addr learned my addr 172.21.15.19:0/3722607561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f4673fff640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674104bf0 0x7f46741024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f467884e640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467410e040 0x7f4674102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f46737fe640 1 -- 172.21.15.19:0/3722607561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467410e040 msgr2=0x7f4674102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f46737fe640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467410e040 0x7f4674102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f46737fe640 1 -- 172.21.15.19:0/3722607561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674104bf0 msgr2=0x7f46741024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.385+0000 7f46737fe640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674104bf0 0x7f46741024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f46737fe640 1 -- 172.21.15.19:0/3722607561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46741a8b10 con 0x7f467410b650 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f467884e640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467410e040 0x7f4674102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f46737fe640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467410b650 0x7f46741029f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f466800b4a0 tx=0x7f466800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f46717fa640 1 -- 172.21.15.19:0/3722607561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668014070 con 0x7f467410b650 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f46717fa640 1 -- 172.21.15.19:0/3722607561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4668010ac0 con 0x7f467410b650 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46741a8e00 con 0x7f467410b650 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f46717fa640 1 -- 172.21.15.19:0/3722607561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668017c30 con 0x7f467410b650 2024-04-25T11:02:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.386+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46741a9270 con 0x7f467410b650 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.388+0000 7f4681238740 1 Processor -- start 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.388+0000 7f4681238740 1 -- start start 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4681238740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a717a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4681238740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a717fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4681238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a7248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4681238740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f4a67dd80 con 0x556f4a34e4d0 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4681238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f4a6378b0 con 0x556f4a3566b0 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4681238740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556f4a679d40 con 0x556f4a377390 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f46717fa640 1 -- 172.21.15.19:0/3722607561 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4668017070 con 0x7f467410b650 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a717fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a717fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58702/0 (socket says 172.21.15.19:58702) 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 -- 172.21.15.19:0/3502154116 learned_addr learned my addr 172.21.15.19:0/3502154116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f467884e640 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a717a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f46737fe640 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a7248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 -- 172.21.15.19:0/3502154116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 msgr2=0x556f4a717a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a717a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 -- 172.21.15.19:0/3502154116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 msgr2=0x556f4a7248f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a7248f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 -- 172.21.15.19:0/3502154116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4a608a40 con 0x556f4a34e4d0 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f46737fe640 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a7248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f4673fff640 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a717fa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4664009900 tx=0x7f4664004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd291728b4f770a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.389+0000 7f46717fa640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f465c0afc60 0x7f465c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.390+0000 7f467884e640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f465c0afc60 0x7f465c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.390+0000 7f465a7fc640 1 -- 172.21.15.19:0/3502154116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664035040 con 0x556f4a34e4d0 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.390+0000 7f465a7fc640 1 -- 172.21.15.19:0/3502154116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4664004770 con 0x556f4a34e4d0 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.390+0000 7f46717fa640 1 -- 172.21.15.19:0/3722607561 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f46680de2c0 con 0x7f467410b650 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.390+0000 7f465a7fc640 1 -- 172.21.15.19:0/3502154116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664003830 con 0x556f4a34e4d0 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 -- 172.21.15.19:0/3502154116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 msgr2=0x556f4a717fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a717fa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4664009900 tx=0x7f4664004390 comp rx=0 tx=0).stop 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f467884e640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f465c0afc60 0x7f465c0b2170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x556f4a714bd0 tx=0x7f466c009580 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 -- 172.21.15.19:0/3502154116 shutdown_connections 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a7248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a717fa0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 --2- 172.21.15.19:0/3502154116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a717a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 -- 172.21.15.19:0/3502154116 >> 172.21.15.19:0/3502154116 conn(0x556f4a36b4e0 msgr2=0x556f4a714210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 -- 172.21.15.19:0/3502154116 shutdown_connections 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 -- 172.21.15.19:0/3502154116 wait complete. 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 -- start start 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.391+0000 7f4681238740 1 Processor -- start 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a34e4d0 0x556f4a722640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a722b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a377390 0x556f4a728440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f4a724f80 con 0x556f4a377390 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f4a604d10 con 0x556f4a34e4d0 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556f4a725100 con 0x556f4a3566b0 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 10 client.0 _ll_register_callbacks cb 0x556f4a631210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a377390 0x556f4a728440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a377390 0x556f4a728440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58706/0 (socket says 172.21.15.19:58706) 2024-04-25T11:02:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 -- 172.21.15.19:0/1997215208 learned_addr learned my addr 172.21.15.19:0/1997215208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4673fff640 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a34e4d0 0x556f4a722640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f46737fe640 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a722b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 -- 172.21.15.19:0/1997215208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 msgr2=0x556f4a722b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a722b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 -- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a34e4d0 msgr2=0x556f4a722640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a34e4d0 0x556f4a722640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 -- 172.21.15.19:0/1997215208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4a4597d0 con 0x556f4a377390 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f467884e640 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a377390 0x556f4a728440 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f466c010900 tx=0x7f466c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6350ca2b0cd01f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f46537fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f46537fe640 1 -- 172.21.15.19:0/1997215208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c004d20 con 0x556f4a377390 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f46537fe640 1 -- 172.21.15.19:0/1997215208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f466c00faf0 con 0x556f4a377390 2024-04-25T11:02:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.392+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4664004770 con 0x556f4a377390 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f46537fe640 1 -- 172.21.15.19:0/1997215208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f466c010e80 con 0x556f4a377390 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 10 client.16806 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556f4a38f3f0 con 0x556f4a377390 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f46537fe640 1 -- 172.21.15.19:0/1997215208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4664004770 con 0x556f4a377390 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f46537fe640 1 -- 172.21.15.19:0/1997215208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f466c00f040 con 0x556f4a377390 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 10 client.16806 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 20 client.16806 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a335f60 0x556f4a7346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 4 client.16806 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f4a68b990 con 0x556f4a335f60 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.393+0000 7f4673fff640 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a335f60 0x556f4a7346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.394+0000 7f4673fff640 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a335f60 0x556f4a7346e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5561d941c2af37 server_cookie=867655c46c1aadc5 in_seq=0 out_seq=0 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.394+0000 7f46537fe640 10 client.16806 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f46537fe640 1 -- 172.21.15.19:0/1997215208 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f4a68b990 con 0x556f4a335f60 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f46537fe640 10 client.16806 handle_command_reply: tid=0 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 client.16806 shutdown 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a377390 msgr2=0x556f4a728440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a377390 0x556f4a728440 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f466c010900 tx=0x7f466c004a40 comp rx=0 tx=0).stop 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 shutdown_connections 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a335f60 0x556f4a7346e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a377390 0x556f4a728440 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a722b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f46537fe640 0 client.16806 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 --2- 172.21.15.19:0/1997215208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a34e4d0 0x556f4a722640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 >> 172.21.15.19:0/1997215208 conn(0x556f4a36b4e0 msgr2=0x556f4a719780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 shutdown_connections 2024-04-25T11:02:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 1 -- 172.21.15.19:0/1997215208 wait complete. 2024-04-25T11:02:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.395+0000 7f4681238740 20 client.16806 trim_cache size 0 max 16384 2024-04-25T11:02:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.413+0000 7f4681238740 1 Processor -- start 2024-04-25T11:02:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.413+0000 7f4681238740 1 -- start start 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.413+0000 7f4681238740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a739ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4681238740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a73a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4681238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a377390 0x556f4a72e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4681238740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46741afc50 con 0x556f4a34e4d0 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4681238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46741b24b0 con 0x556f4a377390 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4681238740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46741b02a0 con 0x556f4a3566b0 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a739ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a739ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58708/0 (socket says 172.21.15.19:58708) 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 -- 172.21.15.19:0/2901172456 learned_addr learned my addr 172.21.15.19:0/2901172456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f46737fe640 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a73a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 -- 172.21.15.19:0/2901172456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 msgr2=0x556f4a73a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a73a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 -- 172.21.15.19:0/2901172456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a377390 msgr2=0x556f4a72e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a377390 0x556f4a72e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 -- 172.21.15.19:0/2901172456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4a34f0f0 con 0x556f4a34e4d0 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4673fff640 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a739ea0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4664005f70 tx=0x7f4664004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fef6a84beb9490b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4659ffb640 1 -- 172.21.15.19:0/2901172456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664035040 con 0x556f4a34e4d0 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4659ffb640 1 -- 172.21.15.19:0/2901172456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46640054b0 con 0x556f4a34e4d0 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.414+0000 7f4659ffb640 1 -- 172.21.15.19:0/2901172456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640314a0 con 0x556f4a34e4d0 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 -- 172.21.15.19:0/2901172456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 msgr2=0x556f4a739ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a739ea0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4664005f70 tx=0x7f4664004580 comp rx=0 tx=0).stop 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 -- 172.21.15.19:0/2901172456 shutdown_connections 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a377390 0x556f4a72e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a3566b0 0x556f4a73a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 --2- 172.21.15.19:0/2901172456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a739ea0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 -- 172.21.15.19:0/2901172456 >> 172.21.15.19:0/2901172456 conn(0x556f4a36b4e0 msgr2=0x556f4a720d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 -- 172.21.15.19:0/2901172456 shutdown_connections 2024-04-25T11:02:37.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 -- 172.21.15.19:0/2901172456 wait complete. 2024-04-25T11:02:37.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.415+0000 7f4681238740 1 -- start start 2024-04-25T11:02:37.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 1 Processor -- start 2024-04-25T11:02:37.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a729050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a729590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a79aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46741b0970 con 0x556f4a34e4d0 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46741b1de0 con 0x556f4a3566b0 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46741b1040 con 0x556f4a377390 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4681238740 10 client.0 _ll_register_callbacks cb 0x7f466c009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a729050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a729050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58722/0 (socket says 172.21.15.19:58722) 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 -- 172.21.15.19:0/2201505808 learned_addr learned my addr 172.21.15.19:0/2201505808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f46737fe640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a729590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f467884e640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a79aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 -- 172.21.15.19:0/2201505808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 msgr2=0x556f4a79aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a79aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 -- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 msgr2=0x556f4a729590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a729590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 -- 172.21.15.19:0/2201505808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4a63e4b0 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f467884e640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a79aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f4673fff640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a729050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f466402a930 tx=0x7f4664005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a70b8e3d35a7be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.416+0000 7f46537fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f46537fe640 1 -- 172.21.15.19:0/2201505808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466405e060 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f46537fe640 1 -- 172.21.15.19:0/2201505808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8513+0+0 (secure 0 0 0) 0x7f4664005170 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46640054b0 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f46537fe640 1 -- 172.21.15.19:0/2201505808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4664035040 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 10 client.16812 fetch_fsmap learned FSMap version 20 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556f4a79be40 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f46537fe640 1 -- 172.21.15.19:0/2201505808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46640054b0 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f46537fe640 1 -- 172.21.15.19:0/2201505808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4664031870 con 0x556f4a34e4d0 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 10 client.16812 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:02:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 20 client.16812 populate_metadata read hostname 'smithi019' 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a30b3a0 0x556f4a79e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 4 client.16812 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.417+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f4a685800 con 0x556f4a30b3a0 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f46737fe640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a30b3a0 0x556f4a79e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f46537fe640 10 client.16812 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f46737fe640 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a30b3a0 0x556f4a79e020 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e191e134db9e5e server_cookie=f382c998d0fb5d25 in_seq=0 out_seq=0 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f46537fe640 1 -- 172.21.15.19:0/2201505808 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556f4a685800 con 0x556f4a30b3a0 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f46537fe640 10 client.16812 handle_command_reply: tid=0 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 client.16812 shutdown 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 msgr2=0x556f4a729050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a729050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f466402a930 tx=0x7f4664005720 comp rx=0 tx=0).stop 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 shutdown_connections 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x556f4a30b3a0 0x556f4a79e020 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f4a377390 0x556f4a79aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f4a3566b0 0x556f4a729590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f46537fe640 0 client.16812 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 --2- 172.21.15.19:0/2201505808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f4a34e4d0 0x556f4a729050 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.418+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 >> 172.21.15.19:0/2201505808 conn(0x556f4a36b4e0 msgr2=0x556f4a7177f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 shutdown_connections 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f4681238740 1 -- 172.21.15.19:0/2201505808 wait complete. 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f4681238740 20 client.16812 trim_cache size 0 max 16384 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:02:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:02:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "ca7c7633-6708-40da-b17d-e46bf31723c8": { 2024-04-25T11:02:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:02:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ca7c7633-6708-40da-b17d-e46bf31723c8", 2024-04-25T11:02:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:02:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:02:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:02:37.427 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f465c0afc60 msgr2=0x7f465c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f465c0afc60 0x7f465c0b2170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x556f4a714bd0 tx=0x7f466c009580 comp rx=0 tx=0).stop 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467410b650 msgr2=0x7f46741029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467410b650 0x7f46741029f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f466800b4a0 tx=0x7f466800b960 comp rx=0 tx=0).stop 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 shutdown_connections 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f465c0afc60 0x7f465c0b2170 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467410e040 0x7f4674102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467410b650 0x7f46741029f0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 --2- 172.21.15.19:0/3722607561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674104bf0 0x7f46741024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 >> 172.21.15.19:0/3722607561 conn(0x7f4674078100 msgr2=0x7f4674101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 shutdown_connections 2024-04-25T11:02:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:02:37.419+0000 7f467a2b1640 1 -- 172.21.15.19:0/3722607561 wait complete. 2024-04-25T11:02:37.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:37 smithi019 ceph-mon[29209]: pgmap v250: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:37.433 INFO:tasks.cephfs.filesystem:scrub status for tag:ca7c7633-6708-40da-b17d-e46bf31723c8 - {'path': '/', 'tag': 'ca7c7633-6708-40da-b17d-e46bf31723c8', 'options': 'recursive,force'} 2024-04-25T11:02:37.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:37 smithi169 ceph-mon[33049]: pgmap v250: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:37.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:37 smithi112 ceph-mon[32799]: pgmap v250: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:38.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:38 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:02:38.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:38 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:38.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:38 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:02:38.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:38 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:38.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:38 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:02:38.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:38 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:38.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:38 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:02:38.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:38 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:38.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:38 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:02:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:38 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:38 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:02:38.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:38 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:02:39.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:39 smithi019 ceph-mon[29209]: pgmap v251: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:39.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:39 smithi169 ceph-mon[33049]: pgmap v251: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:39.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:39 smithi112 ceph-mon[32799]: pgmap v251: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:41.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:41 smithi019 ceph-mon[29209]: pgmap v252: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:41.611 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:41 smithi112 ceph-mon[32799]: pgmap v252: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:41.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:41 smithi169 ceph-mon[33049]: pgmap v252: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:42.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:42 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:02:42.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:42 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:02:42.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:42 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:02:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:43 smithi019 ceph-mon[29209]: pgmap v253: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:02:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:02:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:02:43.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:02:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:43 smithi169 ceph-mon[33049]: pgmap v253: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:02:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:02:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:02:43.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:02:43.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:43 smithi112 ceph-mon[32799]: pgmap v253: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:43.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:02:43.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:02:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:02:43.809 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:02:45.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:45 smithi019 ceph-mon[29209]: pgmap v254: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:45.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:45 smithi169 ceph-mon[33049]: pgmap v254: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:45.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:45 smithi112 ceph-mon[32799]: pgmap v254: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:47.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:47 smithi019 ceph-mon[29209]: pgmap v255: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:47.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:47 smithi169 ceph-mon[33049]: pgmap v255: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:47.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:47 smithi112 ceph-mon[32799]: pgmap v255: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:49.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:49 smithi019 ceph-mon[29209]: pgmap v256: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:49.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:49 smithi169 ceph-mon[33049]: pgmap v256: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:49.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:49 smithi112 ceph-mon[32799]: pgmap v256: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:51.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:51 smithi019 ceph-mon[29209]: pgmap v257: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:51.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:51 smithi169 ceph-mon[33049]: pgmap v257: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:51.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:51 smithi112 ceph-mon[32799]: pgmap v257: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:02:53.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:53 smithi019 ceph-mon[29209]: pgmap v258: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:53.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:53 smithi169 ceph-mon[33049]: pgmap v258: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:53.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:53 smithi112 ceph-mon[32799]: pgmap v258: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:02:55.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:55 smithi019 ceph-mon[29209]: pgmap v259: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:55.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:55 smithi169 ceph-mon[33049]: pgmap v259: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:55.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:55 smithi112 ceph-mon[32799]: pgmap v259: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:57.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:57 smithi019 ceph-mon[29209]: pgmap v260: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:57.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:57 smithi169 ceph-mon[33049]: pgmap v260: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:57.808 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:57 smithi112 ceph-mon[32799]: pgmap v260: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:02:58.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:02:58 smithi019 ceph-mon[29209]: pgmap v261: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:59.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:02:58 smithi169 ceph-mon[33049]: pgmap v261: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:02:59.058 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:02:58 smithi112 ceph-mon[32799]: pgmap v261: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:01.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:01 smithi169 ceph-mon[33049]: pgmap v262: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:01.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:01 smithi112 ceph-mon[32799]: pgmap v262: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:01.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:01 smithi019 ceph-mon[29209]: pgmap v262: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:03.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:03 smithi169 ceph-mon[33049]: pgmap v263: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 33 KiB/s wr, 8 op/s 2024-04-25T11:03:03.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:03 smithi112 ceph-mon[32799]: pgmap v263: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 33 KiB/s wr, 8 op/s 2024-04-25T11:03:03.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:03 smithi019 ceph-mon[29209]: pgmap v263: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 33 KiB/s wr, 8 op/s 2024-04-25T11:03:05.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:05 smithi169 ceph-mon[33049]: pgmap v264: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 33 KiB/s wr, 7 op/s 2024-04-25T11:03:05.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:05 smithi112 ceph-mon[32799]: pgmap v264: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 33 KiB/s wr, 7 op/s 2024-04-25T11:03:05.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:05 smithi019 ceph-mon[29209]: pgmap v264: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 33 KiB/s wr, 7 op/s 2024-04-25T11:03:07.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:03:07.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:03:07.462 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:07 smithi019 ceph-mon[29209]: pgmap v265: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:03:07.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:07 smithi169 ceph-mon[33049]: pgmap v265: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:03:07.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:07 smithi112 ceph-mon[32799]: pgmap v265: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:03:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 -- 172.21.15.19:0/299526791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c68104990 msgr2=0x7f4c68101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 --2- 172.21.15.19:0/299526791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c68104990 0x7f4c68101b70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009900 tx=0x7f4c6402a480 comp rx=0 tx=0).stop 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 -- 172.21.15.19:0/299526791 shutdown_connections 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 --2- 172.21.15.19:0/299526791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c681102f0 0x7f4c68112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 --2- 172.21.15.19:0/299526791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c681020b0 0x7f4c681024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 --2- 172.21.15.19:0/299526791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c68104990 0x7f4c68101b70 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 -- 172.21.15.19:0/299526791 >> 172.21.15.19:0/299526791 conn(0x7f4c68077880 msgr2=0x7f4c68077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 -- 172.21.15.19:0/299526791 shutdown_connections 2024-04-25T11:03:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.567+0000 7f4c70164640 1 -- 172.21.15.19:0/299526791 wait complete. 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 Processor -- start 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 -- start start 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c681020b0 0x7f4c681a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c68104990 0x7f4c681a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681102f0 0x7f4c6819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68115ee0 con 0x7f4c681102f0 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68115d60 con 0x7f4c68104990 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c70164640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68116060 con 0x7f4c681020b0 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c6e701640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681102f0 0x7f4c6819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c6e701640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681102f0 0x7f4c6819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59128/0 (socket says 172.21.15.19:59128) 2024-04-25T11:03:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.568+0000 7f4c6e701640 1 -- 172.21.15.19:0/1329617678 learned_addr learned my addr 172.21.15.19:0/1329617678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c68104990 0x7f4c681a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c681020b0 0x7f4c681a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6e701640 1 -- 172.21.15.19:0/1329617678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c681020b0 msgr2=0x7f4c681a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6e701640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c681020b0 0x7f4c681a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6e701640 1 -- 172.21.15.19:0/1329617678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c68104990 msgr2=0x7f4c681a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6e701640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c68104990 0x7f4c681a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6e701640 1 -- 172.21.15.19:0/1329617678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6819cea0 con 0x7f4c681102f0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c6e701640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681102f0 0x7f4c6819c7c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000eb40 tx=0x7f4c6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c5effd640 1 -- 172.21.15.19:0/1329617678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60004b20 con 0x7f4c681102f0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c5effd640 1 -- 172.21.15.19:0/1329617678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c60004cc0 con 0x7f4c681102f0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6819d190 con 0x7f4c681102f0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c5effd640 1 -- 172.21.15.19:0/1329617678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60010720 con 0x7f4c681102f0 2024-04-25T11:03:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.569+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c681a93e0 con 0x7f4c681102f0 2024-04-25T11:03:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.572+0000 7f4c77013740 1 Processor -- start 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c5effd640 1 -- 172.21.15.19:0/1329617678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4c60005560 con 0x7f4c681102f0 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c77013740 1 -- start start 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c77013740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 0x56374fda6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c77013740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fda6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c77013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdb38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c77013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56374fd0cd80 con 0x56374f9e56b0 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c77013740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56374fcc68b0 con 0x56374f9dd4d0 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c77013740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56374fd08d40 con 0x56374fa06390 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c6d6ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdb38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c6d6ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdb38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59138/0 (socket says 172.21.15.19:59138) 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1867193090 learned_addr learned my addr 172.21.15.19:0/1867193090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c5effd640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4c540afc60 0x7f4c540b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 0x56374fda6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.573+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fda6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6e701640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4c540afc60 0x7f4c540b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1867193090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 msgr2=0x56374fda6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 0x56374fda6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1867193090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 msgr2=0x56374fda6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c5effd640 1 -- 172.21.15.19:0/1329617678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f4c600df780 con 0x7f4c681102f0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fda6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1867193090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374fc97a40 con 0x56374f9e56b0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdb38f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009fd0 tx=0x7f4c58005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a78ea1966d2c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c6e701640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4c540afc60 0x7f4c540b2170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c600a3980 tx=0x7f4c60075160 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c53fff640 1 -- 172.21.15.19:0/1867193090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58004030 con 0x56374f9e56b0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c53fff640 1 -- 172.21.15.19:0/1867193090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5800fc40 con 0x56374f9e56b0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.574+0000 7f4c53fff640 1 -- 172.21.15.19:0/1867193090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c580037f0 con 0x56374f9e56b0 2024-04-25T11:03:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 -- 172.21.15.19:0/1867193090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 msgr2=0x56374fdb38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdb38f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009fd0 tx=0x7f4c58005850 comp rx=0 tx=0).stop 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 -- 172.21.15.19:0/1867193090 shutdown_connections 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdb38f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fda6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 --2- 172.21.15.19:0/1867193090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 0x56374fda6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 -- 172.21.15.19:0/1867193090 >> 172.21.15.19:0/1867193090 conn(0x56374f9fa4e0 msgr2=0x56374fda3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 -- 172.21.15.19:0/1867193090 shutdown_connections 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.575+0000 7f4c77013740 1 -- 172.21.15.19:0/1867193090 wait complete. 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 -- start start 2024-04-25T11:03:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 Processor -- start 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 0x56374fdbd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56374fdb3fe0 con 0x56374f9dd4d0 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56374fc93d10 con 0x56374fa06390 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56374fdb4160 con 0x56374f9e56b0 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c77013740 10 client.0 _ll_register_callbacks cb 0x56374fcc0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c6d6ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c6d6ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59144/0 (socket says 172.21.15.19:59144) 2024-04-25T11:03:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.576+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/3899079819 learned_addr learned my addr 172.21.15.19:0/3899079819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c6df00640 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/3899079819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 msgr2=0x56374fdb1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/3899079819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 msgr2=0x56374fdbd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 0x56374fdbd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/3899079819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374fae87d0 con 0x56374f9dd4d0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb1570 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4c580045a0 tx=0x7f4c58005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3bff2292e6c50f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c517fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c517fa640 1 -- 172.21.15.19:0/3899079819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58005030 con 0x56374f9dd4d0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c517fa640 1 -- 172.21.15.19:0/3899079819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f4c580051d0 con 0x56374f9dd4d0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5800fc40 con 0x56374f9dd4d0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c517fa640 1 -- 172.21.15.19:0/3899079819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4c580053d0 con 0x56374f9dd4d0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c77013740 10 client.16824 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56374fa1e3f0 con 0x56374f9dd4d0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.577+0000 7f4c517fa640 1 -- 172.21.15.19:0/3899079819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5800fc40 con 0x56374f9dd4d0 2024-04-25T11:03:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c517fa640 1 -- 172.21.15.19:0/3899079819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4c5801f660 con 0x56374f9dd4d0 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c77013740 10 client.16824 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c77013740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c77013740 20 client.16824 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c77013740 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f9c4f60 0x56374fdc34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c77013740 4 client.16824 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56374fd1a990 con 0x56374f9c4f60 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.578+0000 7f4c6df00640 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f9c4f60 0x56374fdc34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.579+0000 7f4c517fa640 10 client.16824 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.579+0000 7f4c6df00640 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f9c4f60 0x56374fdc34b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8fc7ed00e1e4a8 server_cookie=d1a06a5d2ecfbd65 in_seq=0 out_seq=0 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.579+0000 7f4c517fa640 1 -- 172.21.15.19:0/3899079819 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56374fd1a990 con 0x56374f9c4f60 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.579+0000 7f4c517fa640 10 client.16824 handle_command_reply: tid=0 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.579+0000 7f4c77013740 1 client.16824 shutdown 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.579+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 msgr2=0x56374fdb1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb1570 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4c580045a0 tx=0x7f4c58005730 comp rx=0 tx=0).stop 2024-04-25T11:03:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 shutdown_connections 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f9c4f60 0x56374fdc34b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 0x56374fdbd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 --2- 172.21.15.19:0/3899079819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb1570 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c517fa640 0 client.16824 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 >> 172.21.15.19:0/3899079819 conn(0x56374f9fa4e0 msgr2=0x56374fda8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 shutdown_connections 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 1 -- 172.21.15.19:0/3899079819 wait complete. 2024-04-25T11:03:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.580+0000 7f4c77013740 20 client.16824 trim_cache size 0 max 16384 2024-04-25T11:03:07.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 Processor -- start 2024-04-25T11:03:07.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 -- start start 2024-04-25T11:03:07.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fdb0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdc8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 0x56374fdc8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681afc50 con 0x56374f9e56b0 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b24b0 con 0x56374f9dd4d0 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c77013740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b02a0 con 0x56374fa06390 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.598+0000 7f4c6df00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdc8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdc8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59150/0 (socket says 172.21.15.19:59150) 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 -- 172.21.15.19:0/1883827525 learned_addr learned my addr 172.21.15.19:0/1883827525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fdb0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 -- 172.21.15.19:0/1883827525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 msgr2=0x56374fdc8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 0x56374fdc8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 -- 172.21.15.19:0/1883827525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 msgr2=0x56374fdb0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fdb0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 -- 172.21.15.19:0/1883827525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374f9de0f0 con 0x56374f9e56b0 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdc8a80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4c6402a9a0 tx=0x7f4c64031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b55c3d34a08a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c537fe640 1 -- 172.21.15.19:0/1883827525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6404b070 con 0x56374f9e56b0 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c537fe640 1 -- 172.21.15.19:0/1883827525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c640054f0 con 0x56374f9e56b0 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.599+0000 7f4c537fe640 1 -- 172.21.15.19:0/1883827525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64035040 con 0x56374f9e56b0 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 -- 172.21.15.19:0/1883827525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 msgr2=0x56374fdc8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdc8a80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4c6402a9a0 tx=0x7f4c64031400 comp rx=0 tx=0).stop 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 -- 172.21.15.19:0/1883827525 shutdown_connections 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374fa06390 0x56374fdc8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9e56b0 0x56374fdc8a80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 --2- 172.21.15.19:0/1883827525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374f9dd4d0 0x56374fdb0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 -- 172.21.15.19:0/1883827525 >> 172.21.15.19:0/1883827525 conn(0x56374f9fa4e0 msgr2=0x56374fdafb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 -- 172.21.15.19:0/1883827525 shutdown_connections 2024-04-25T11:03:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 -- 172.21.15.19:0/1883827525 wait complete. 2024-04-25T11:03:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 -- start start 2024-04-25T11:03:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.600+0000 7f4c77013740 1 Processor -- start 2024-04-25T11:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c77013740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c77013740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c77013740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 0x56374fdb9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c77013740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b0970 con 0x56374f9dd4d0 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c77013740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b1de0 con 0x56374fa06390 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c77013740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b1040 con 0x56374f9e56b0 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c77013740 10 client.0 _ll_register_callbacks cb 0x7f4c600093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59156/0 (socket says 172.21.15.19:59156) 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1547650769 learned_addr learned my addr 172.21.15.19:0/1547650769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 0x56374fdb9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1547650769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 msgr2=0x56374fdb5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 msgr2=0x56374fdb9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 0x56374fdb9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.601+0000 7f4c6d6ff640 1 -- 172.21.15.19:0/1547650769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374fccd4b0 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c6d6ff640 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb4c50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4c580057c0 tx=0x7f4c580045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704b6a990d5ad6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c517fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c517fa640 1 -- 172.21.15.19:0/1547650769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58036040 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c517fa640 1 -- 172.21.15.19:0/1547650769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f4c5802c0b0 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c640054f0 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c517fa640 1 -- 172.21.15.19:0/1547650769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4c58045050 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 10 client.16830 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56374fe2aef0 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c517fa640 1 -- 172.21.15.19:0/1547650769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c640054f0 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c517fa640 1 -- 172.21.15.19:0/1547650769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4c5803c070 con 0x56374f9dd4d0 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 10 client.16830 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 20 client.16830 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f99a3a0 0x56374fe2d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 4 client.16830 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.602+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56374fd14800 con 0x56374f99a3a0 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f99a3a0 0x56374fe2d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c517fa640 10 client.16830 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c6df00640 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f99a3a0 0x56374fe2d5a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d84f9ef14573442 server_cookie=a2e72342dddded02 in_seq=0 out_seq=0 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c517fa640 1 -- 172.21.15.19:0/1547650769 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56374fd14800 con 0x56374f99a3a0 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c517fa640 10 client.16830 handle_command_reply: tid=0 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 client.16830 shutdown 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 msgr2=0x56374fdb4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb4c50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4c580057c0 tx=0x7f4c580045d0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 shutdown_connections 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56374f99a3a0 0x56374fe2d5a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56374fa06390 0x56374fdb9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56374f9e56b0 0x56374fdb5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 --2- 172.21.15.19:0/1547650769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374f9dd4d0 0x56374fdb4c50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c517fa640 0 client.16830 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.603+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 >> 172.21.15.19:0/1547650769 conn(0x56374f9fa4e0 msgr2=0x56374fda19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 shutdown_connections 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c77013740 1 -- 172.21.15.19:0/1547650769 wait complete. 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c77013740 20 client.16830 trim_cache size 0 max 16384 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:03:07.610 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:03:07.611 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:03:07.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4c540afc60 msgr2=0x7f4c540b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4c540afc60 0x7f4c540b2170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c600a3980 tx=0x7f4c60075160 comp rx=0 tx=0).stop 2024-04-25T11:03:07.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681102f0 msgr2=0x7f4c6819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681102f0 0x7f4c6819c7c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000eb40 tx=0x7f4c6000c2d0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 shutdown_connections 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f4c540afc60 0x7f4c540b2170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681102f0 0x7f4c6819c7c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c68104990 0x7f4c681a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 --2- 172.21.15.19:0/1329617678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c681020b0 0x7f4c681a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 >> 172.21.15.19:0/1329617678 conn(0x7f4c68077880 msgr2=0x7f4c68100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 shutdown_connections 2024-04-25T11:03:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.604+0000 7f4c70164640 1 -- 172.21.15.19:0/1329617678 wait complete. 2024-04-25T11:03:07.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:03:07.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:03:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 -- 172.21.15.19:0/48440731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 msgr2=0x7f8ab4102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 --2- 172.21.15.19:0/48440731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4102620 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa802a400 comp rx=0 tx=0).stop 2024-04-25T11:03:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 -- 172.21.15.19:0/48440731 shutdown_connections 2024-04-25T11:03:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 --2- 172.21.15.19:0/48440731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab410d3a0 0x7f8ab410f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 --2- 172.21.15.19:0/48440731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab4102b60 0x7f8ab4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 --2- 172.21.15.19:0/48440731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4102620 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 -- 172.21.15.19:0/48440731 >> 172.21.15.19:0/48440731 conn(0x7f8ab4078040 msgr2=0x7f8ab4078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 -- 172.21.15.19:0/48440731 shutdown_connections 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.769+0000 7f8abab4d640 1 -- 172.21.15.19:0/48440731 wait complete. 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 Processor -- start 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 -- start start 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4102b60 0x7f8ab41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab410d3a0 0x7f8ab419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111b40 con 0x7f8ab4108a40 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41119c0 con 0x7f8ab410d3a0 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.770+0000 7f8abab4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111cc0 con 0x7f8ab4102b60 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59188/0 (socket says 172.21.15.19:59188) 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 -- 172.21.15.19:0/3481040936 learned_addr learned my addr 172.21.15.19:0/3481040936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab90ea640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab410d3a0 0x7f8ab419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab88e9640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4102b60 0x7f8ab41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 -- 172.21.15.19:0/3481040936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4102b60 msgr2=0x7f8ab41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4102b60 0x7f8ab41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 -- 172.21.15.19:0/3481040936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab410d3a0 msgr2=0x7f8ab419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab410d3a0 0x7f8ab419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 -- 172.21.15.19:0/3481040936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab419d320 con 0x7f8ab4108a40 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab3fff640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4198a20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009c10 tx=0x7f8aa0007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab1ffb640 1 -- 172.21.15.19:0/3481040936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0009c40 con 0x7f8ab4108a40 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab1ffb640 1 -- 172.21.15.19:0/3481040936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa0009de0 con 0x7f8ab4108a40 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8ab1ffb640 1 -- 172.21.15.19:0/3481040936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00193f0 con 0x7f8ab4108a40 2024-04-25T11:03:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.771+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab419d610 con 0x7f8ab4108a40 2024-04-25T11:03:07.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.772+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a4f90 con 0x7f8ab4108a40 2024-04-25T11:03:07.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.773+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab406a190 con 0x7f8ab4108a40 2024-04-25T11:03:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.774+0000 7f8ab1ffb640 1 -- 172.21.15.19:0/3481040936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8aa0020030 con 0x7f8ab4108a40 2024-04-25T11:03:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.775+0000 7f8ab1ffb640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8aa40afc60 0x7f8aa40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.775+0000 7f8ab88e9640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8aa40afc60 0x7f8aa40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.775+0000 7f8ab1ffb640 1 -- 172.21.15.19:0/3481040936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f8aa00e2630 con 0x7f8ab4108a40 2024-04-25T11:03:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.775+0000 7f8ab88e9640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8aa40afc60 0x7f8aa40b2170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8031440 tx=0x7f8aa8005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:07.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:07.780+0000 7f8ab1ffb640 1 -- 172.21.15.19:0/3481040936 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8aa0027060 con 0x7f8ab4108a40 2024-04-25T11:03:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.007+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ab4106f00 con 0x7f8ab4108a40 2024-04-25T11:03:08.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.008+0000 7f8ab1ffb640 1 -- 172.21.15.19:0/3481040936 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f8aa0016070 con 0x7f8ab4108a40 2024-04-25T11:03:08.008 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:03:08.008 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:03:08.009 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:03:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8aa40afc60 msgr2=0x7f8aa40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8aa40afc60 0x7f8aa40b2170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8031440 tx=0x7f8aa8005a20 comp rx=0 tx=0).stop 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 msgr2=0x7f8ab4198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4198a20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009c10 tx=0x7f8aa0007590 comp rx=0 tx=0).stop 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 shutdown_connections 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8aa40afc60 0x7f8aa40b2170 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab410d3a0 0x7f8ab419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4108a40 0x7f8ab4198a20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 --2- 172.21.15.19:0/3481040936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4102b60 0x7f8ab41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 >> 172.21.15.19:0/3481040936 conn(0x7f8ab4078040 msgr2=0x7f8ab41040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 shutdown_connections 2024-04-25T11:03:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.010+0000 7f8abab4d640 1 -- 172.21.15.19:0/3481040936 wait complete. 2024-04-25T11:03:08.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:03:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 -- 172.21.15.19:0/930648407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 msgr2=0x7fc02410fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/930648407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc02410fe50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01802a480 comp rx=0 tx=0).stop 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 -- 172.21.15.19:0/930648407 shutdown_connections 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/930648407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc024110390 0x7fc0241127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/930648407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc02410fe50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/930648407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0240763f0 0x7fc024076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 -- 172.21.15.19:0/930648407 >> 172.21.15.19:0/930648407 conn(0x7fc0240ff1e0 msgr2=0x7fc024101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 -- 172.21.15.19:0/930648407 shutdown_connections 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 -- 172.21.15.19:0/930648407 wait complete. 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 Processor -- start 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.136+0000 7fc02c2bd640 1 -- start start 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02c2bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0240763f0 0x7fc0241a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02c2bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc0241a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02c2bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc024110390 0x7fc02419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02c2bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc024115f00 con 0x7fc02410da30 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02c2bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc024115d80 con 0x7fc024110390 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02c2bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc024116080 con 0x7fc0240763f0 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc0241a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc0241a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59208/0 (socket says 172.21.15.19:59208) 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 -- 172.21.15.19:0/1381911766 learned_addr learned my addr 172.21.15.19:0/1381911766 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02a85a640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc024110390 0x7fc02419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02a059640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0240763f0 0x7fc0241a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 -- 172.21.15.19:0/1381911766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0240763f0 msgr2=0x7fc0241a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0240763f0 0x7fc0241a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 -- 172.21.15.19:0/1381911766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc024110390 msgr2=0x7fc02419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc024110390 0x7fc02419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 -- 172.21.15.19:0/1381911766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02419ce50 con 0x7fc02410da30 2024-04-25T11:03:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02a85a640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc024110390 0x7fc02419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc02a059640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0240763f0 0x7fc0241a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.137+0000 7fc029858640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc0241a2af0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc018031a90 tx=0x7fc018004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.138+0000 7fc0237fe640 1 -- 172.21.15.19:0/1381911766 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01803f030 con 0x7fc02410da30 2024-04-25T11:03:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.138+0000 7fc0237fe640 1 -- 172.21.15.19:0/1381911766 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018003cb0 con 0x7fc02410da30 2024-04-25T11:03:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.138+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02419d0e0 con 0x7fc02410da30 2024-04-25T11:03:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.138+0000 7fc0237fe640 1 -- 172.21.15.19:0/1381911766 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035070 con 0x7fc02410da30 2024-04-25T11:03:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.138+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02419d5d0 con 0x7fc02410da30 2024-04-25T11:03:08.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.140+0000 7fc033172740 1 Processor -- start 2024-04-25T11:03:08.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc033172740 1 -- start start 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc0237fe640 1 -- 172.21.15.19:0/1381911766 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc018042030 con 0x7fc02410da30 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc033172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae685b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc033172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae685bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc033172740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae68686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc033172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ae67c0d80 con 0x563ae64ba390 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc033172740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ae677aa20 con 0x563ae64996b0 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc033172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ae67bcd40 con 0x563ae64914d0 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae685b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae685b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59214/0 (socket says 172.21.15.19:59214) 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 -- 172.21.15.19:0/2697928225 learned_addr learned my addr 172.21.15.19:0/2697928225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc029858640 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae68686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a059640 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae685bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 -- 172.21.15.19:0/2697928225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 msgr2=0x563ae685bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae685bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 -- 172.21.15.19:0/2697928225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 msgr2=0x563ae68686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae68686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.141+0000 7fc02a85a640 1 -- 172.21.15.19:0/2697928225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae6785c30 con 0x563ae64ba390 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.142+0000 7fc02a85a640 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae685b810 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc014008de0 tx=0x7fc01400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50260862a17d34e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.142+0000 7fc0237fe640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc00c0afc60 0x7fc00c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.142+0000 7fc02a059640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc00c0afc60 0x7fc00c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.142+0000 7fc0237fe640 1 -- 172.21.15.19:0/1381911766 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7fc0180ff700 con 0x7fc02410da30 2024-04-25T11:03:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.142+0000 7fc02a059640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc00c0afc60 0x7fc00c0b2170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x563ae685cfe0 tx=0x7fc01c00b420 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:08.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.144+0000 7fc020bfe640 1 -- 172.21.15.19:0/2697928225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01400f830 con 0x563ae64ba390 2024-04-25T11:03:08.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.144+0000 7fc020bfe640 1 -- 172.21.15.19:0/2697928225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01400de20 con 0x563ae64ba390 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.144+0000 7fc020bfe640 1 -- 172.21.15.19:0/2697928225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01400e3b0 con 0x563ae64ba390 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 -- 172.21.15.19:0/2697928225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 msgr2=0x563ae685b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae685b810 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc014008de0 tx=0x7fc01400d710 comp rx=0 tx=0).stop 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 -- 172.21.15.19:0/2697928225 shutdown_connections 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae68686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae685bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 --2- 172.21.15.19:0/2697928225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae685b810 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 -- 172.21.15.19:0/2697928225 >> 172.21.15.19:0/2697928225 conn(0x563ae64ae4e0 msgr2=0x563ae6857fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 -- 172.21.15.19:0/2697928225 shutdown_connections 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 -- 172.21.15.19:0/2697928225 wait complete. 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.145+0000 7fc033172740 1 -- start start 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 1 Processor -- start 2024-04-25T11:03:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6866810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae6872250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ae6868d90 con 0x563ae64914d0 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ae6747dc0 con 0x563ae64ba390 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ae6868f10 con 0x563ae64996b0 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc033172740 10 client.0 _ll_register_callbacks cb 0x563ae6763600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68662d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59228/0 (socket says 172.21.15.19:59228) 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 -- 172.21.15.19:0/904008676 learned_addr learned my addr 172.21.15.19:0/904008676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a85a640 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae6872250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc029858640 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6866810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 -- 172.21.15.19:0/904008676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 msgr2=0x563ae6866810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6866810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 -- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 msgr2=0x563ae6872250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae6872250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.146+0000 7fc02a059640 1 -- 172.21.15.19:0/904008676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae64d23f0 con 0x563ae64914d0 2024-04-25T11:03:08.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.147+0000 7fc02a059640 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68662d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc01c01bce0 tx=0x7fc01c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2af787c51640280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:08.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.147+0000 7fc009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc009ffb640 1 -- 172.21.15.19:0/904008676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00ecc0 con 0x563ae64914d0 2024-04-25T11:03:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc009ffb640 1 -- 172.21.15.19:0/904008676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7fc01c027480 con 0x563ae64914d0 2024-04-25T11:03:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01400de20 con 0x563ae64914d0 2024-04-25T11:03:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc009ffb640 1 -- 172.21.15.19:0/904008676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc01c00ee60 con 0x563ae64914d0 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 10 client.16848 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563ae659c7d0 con 0x563ae64914d0 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc009ffb640 1 -- 172.21.15.19:0/904008676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01400de20 con 0x563ae64914d0 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc009ffb640 1 -- 172.21.15.19:0/904008676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc01c01c610 con 0x563ae64914d0 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 10 client.16848 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 20 client.16848 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae6478f60 0x563ae68787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 4 client.16848 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.149+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae67ce990 con 0x563ae6478f60 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.150+0000 7fc029858640 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae6478f60 0x563ae68787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.150+0000 7fc009ffb640 10 client.16848 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.150+0000 7fc029858640 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae6478f60 0x563ae68787f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e4ece8db686ddb server_cookie=4dfe011f86041db in_seq=0 out_seq=0 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc009ffb640 1 -- 172.21.15.19:0/904008676 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ae67ce990 con 0x563ae6478f60 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc009ffb640 10 client.16848 handle_command_reply: tid=0 2024-04-25T11:03:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 client.16848 shutdown 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 msgr2=0x563ae68662d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68662d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc01c01bce0 tx=0x7fc01c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 shutdown_connections 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae6478f60 0x563ae68787f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae6872250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6866810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc009ffb640 0 client.16848 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 --2- 172.21.15.19:0/904008676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68662d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 >> 172.21.15.19:0/904008676 conn(0x563ae64ae4e0 msgr2=0x563ae685d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 shutdown_connections 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 1 -- 172.21.15.19:0/904008676 wait complete. 2024-04-25T11:03:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.151+0000 7fc033172740 20 client.16848 trim_cache size 0 max 16384 2024-04-25T11:03:08.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 Processor -- start 2024-04-25T11:03:08.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 -- start start 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae6871e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae6856d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae6857260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241afc70 con 0x563ae64ba390 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b24d0 con 0x563ae64996b0 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.165+0000 7fc033172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b02c0 con 0x563ae64914d0 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae6857260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae6857260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59234/0 (socket says 172.21.15.19:59234) 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 -- 172.21.15.19:0/3959140142 learned_addr learned my addr 172.21.15.19:0/3959140142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a059640 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae6871e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc029858640 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae6856d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 -- 172.21.15.19:0/3959140142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 msgr2=0x563ae6871e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae6871e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 -- 172.21.15.19:0/3959140142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 msgr2=0x563ae6856d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae6856d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 -- 172.21.15.19:0/3959140142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae6861c80 con 0x563ae64ba390 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc029858640 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae6856d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc02a85a640 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae6857260 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc014008d00 tx=0x7fc0140089b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e1eb109699cda8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc00bfff640 1 -- 172.21.15.19:0/3959140142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014018dd0 con 0x563ae64ba390 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc00bfff640 1 -- 172.21.15.19:0/3959140142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc014018480 con 0x563ae64ba390 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.166+0000 7fc00bfff640 1 -- 172.21.15.19:0/3959140142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01400f9b0 con 0x563ae64ba390 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 -- 172.21.15.19:0/3959140142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 msgr2=0x563ae6857260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae6857260 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc014008d00 tx=0x7fc0140089b0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 -- 172.21.15.19:0/3959140142 shutdown_connections 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64ba390 0x563ae6857260 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64996b0 0x563ae6856d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 --2- 172.21.15.19:0/3959140142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64914d0 0x563ae6871e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 -- 172.21.15.19:0/3959140142 >> 172.21.15.19:0/3959140142 conn(0x563ae64ae4e0 msgr2=0x563ae6856610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 -- 172.21.15.19:0/3959140142 shutdown_connections 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 -- 172.21.15.19:0/3959140142 wait complete. 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.167+0000 7fc033172740 1 -- start start 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 1 Processor -- start 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6869d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae68d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b0990 con 0x563ae64914d0 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b1e00 con 0x563ae64ba390 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b1060 con 0x563ae64996b0 2024-04-25T11:03:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc033172740 10 client.0 _ll_register_callbacks cb 0x7fc01c00b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59238/0 (socket says 172.21.15.19:59238) 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 -- 172.21.15.19:0/3267427398 learned_addr learned my addr 172.21.15.19:0/3267427398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a85a640 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae68d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc029858640 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6869d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 -- 172.21.15.19:0/3267427398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 msgr2=0x563ae6869d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6869d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 -- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 msgr2=0x563ae68d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae68d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.168+0000 7fc02a059640 1 -- 172.21.15.19:0/3267427398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae68d9140 con 0x563ae64914d0 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc02a059640 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68697d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0058c0 tx=0x7fc01c01c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834ca1682ac13249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc009ffb640 1 -- 172.21.15.19:0/3267427398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c03e060 con 0x563ae64914d0 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc009ffb640 1 -- 172.21.15.19:0/3267427398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7fc01c027680 con 0x563ae64914d0 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc014018480 con 0x563ae64914d0 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc009ffb640 1 -- 172.21.15.19:0/3267427398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc01c033040 con 0x563ae64914d0 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 10 client.16854 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563ae68e0460 con 0x563ae64914d0 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc009ffb640 1 -- 172.21.15.19:0/3267427398 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc014018480 con 0x563ae64914d0 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc009ffb640 1 -- 172.21.15.19:0/3267427398 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc01c036070 con 0x563ae64914d0 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 10 client.16854 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 20 client.16854 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae644e3a0 0x563ae68e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 4 client.16854 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:03:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.169+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ae67c8800 con 0x563ae644e3a0 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc029858640 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae644e3a0 0x563ae68e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc029858640 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae644e3a0 0x563ae68e2bf0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37562ae116da4419 server_cookie=2d39c462a231dda9 in_seq=0 out_seq=0 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc009ffb640 10 client.16854 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc009ffb640 1 -- 172.21.15.19:0/3267427398 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ae67c8800 con 0x563ae644e3a0 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc009ffb640 10 client.16854 handle_command_reply: tid=0 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 client.16854 shutdown 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 msgr2=0x563ae68697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68697d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0058c0 tx=0x7fc01c01c6c0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 shutdown_connections 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563ae644e3a0 0x563ae68e2bf0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae64ba390 0x563ae68d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae64996b0 0x563ae6869d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 --2- 172.21.15.19:0/3267427398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae64914d0 0x563ae68697d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc009ffb640 0 client.16854 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 >> 172.21.15.19:0/3267427398 conn(0x563ae64ae4e0 msgr2=0x563ae6857040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 shutdown_connections 2024-04-25T11:03:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.170+0000 7fc033172740 1 -- 172.21.15.19:0/3267427398 wait complete. 2024-04-25T11:03:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc033172740 20 client.16854 trim_cache size 0 max 16384 2024-04-25T11:03:08.178 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc00c0afc60 msgr2=0x7fc00c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc00c0afc60 0x7fc00c0b2170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x563ae685cfe0 tx=0x7fc01c00b420 comp rx=0 tx=0).stop 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 msgr2=0x7fc0241a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc0241a2af0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc018031a90 tx=0x7fc018004270 comp rx=0 tx=0).stop 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 shutdown_connections 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc00c0afc60 0x7fc00c0b2170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc024110390 0x7fc02419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc02410da30 0x7fc0241a2af0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 --2- 172.21.15.19:0/1381911766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0240763f0 0x7fc0241a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 >> 172.21.15.19:0/1381911766 conn(0x7fc0240ff1e0 msgr2=0x7fc024110820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 shutdown_connections 2024-04-25T11:03:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:08.171+0000 7fc02c2bd640 1 -- 172.21.15.19:0/1381911766 wait complete. 2024-04-25T11:03:08.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:08 smithi019 ceph-mon[29209]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:03:08.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:08 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3481040936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:03:08.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:08 smithi169 ceph-mon[33049]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:03:08.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:08 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3481040936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:03:08.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:08 smithi112 ceph-mon[32799]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:03:08.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:08 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3481040936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:03:09.183 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:03:09.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:03:09.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:03:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 -- 172.21.15.19:0/1986982948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81031d0 msgr2=0x7f14d810f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 --2- 172.21.15.19:0/1986982948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81031d0 0x7f14d810f4b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009ca0 tx=0x7f14d002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 -- 172.21.15.19:0/1986982948 shutdown_connections 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 --2- 172.21.15.19:0/1986982948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81031d0 0x7f14d810f4b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 --2- 172.21.15.19:0/1986982948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d8108ec0 0x7f14d8102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 --2- 172.21.15.19:0/1986982948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d8108a20 0x7f14d8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 -- 172.21.15.19:0/1986982948 >> 172.21.15.19:0/1986982948 conn(0x7f14d8078180 msgr2=0x7f14d80785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 -- 172.21.15.19:0/1986982948 shutdown_connections 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 -- 172.21.15.19:0/1986982948 wait complete. 2024-04-25T11:03:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.315+0000 7f14de71f640 1 Processor -- start 2024-04-25T11:03:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14de71f640 1 -- start start 2024-04-25T11:03:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14de71f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d81031d0 0x7f14d819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14de71f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8108a20 0x7f14d819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14de71f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d8108ec0 0x7f14d819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14de71f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8113d40 con 0x7f14d8108a20 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14de71f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8113bc0 con 0x7f14d8108ec0 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14de71f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8113ec0 con 0x7f14d81031d0 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8108a20 0x7f14d819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8108a20 0x7f14d819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59258/0 (socket says 172.21.15.19:59258) 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14d77fe640 1 -- 172.21.15.19:0/2760584050 learned_addr learned my addr 172.21.15.19:0/2760584050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d81031d0 0x7f14d819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.316+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d8108ec0 0x7f14d819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d77fe640 1 -- 172.21.15.19:0/2760584050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d81031d0 msgr2=0x7f14d819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d81031d0 0x7f14d819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d77fe640 1 -- 172.21.15.19:0/2760584050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d8108ec0 msgr2=0x7f14d819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d8108ec0 0x7f14d819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d77fe640 1 -- 172.21.15.19:0/2760584050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d819f4a0 con 0x7f14d8108a20 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d8108ec0 0x7f14d819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8108a20 0x7f14d819ac20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00d770 tx=0x7f14cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d57fa640 1 -- 172.21.15.19:0/2760584050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc017070 con 0x7f14d8108a20 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d57fa640 1 -- 172.21.15.19:0/2760584050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc00de80 con 0x7f14d8108a20 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d819f790 con 0x7f14d8108a20 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14d57fa640 1 -- 172.21.15.19:0/2760584050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc003c20 con 0x7f14d8108a20 2024-04-25T11:03:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.317+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d819fce0 con 0x7f14d8108a20 2024-04-25T11:03:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 Processor -- start 2024-04-25T11:03:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d57fa640 1 -- 172.21.15.19:0/2760584050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f14cc003dc0 con 0x7f14d8108a20 2024-04-25T11:03:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 -- start start 2024-04-25T11:03:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4e7550 0x557eaa8b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557eaa816b00 con 0x557eaa4e7550 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557eaa7b2920 con 0x557eaa4ef730 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14e55ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557eaa812ac0 con 0x557eaa510410 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4e7550 0x557eaa8b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4e7550 0x557eaa8b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59274/0 (socket says 172.21.15.19:59274) 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 -- 172.21.15.19:0/4022322406 learned_addr learned my addr 172.21.15.19:0/4022322406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d77fe640 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14dccbc640 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 -- 172.21.15.19:0/4022322406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 msgr2=0x557eaa8b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d57fa640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f14c00afc60 0x7f14c00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 -- 172.21.15.19:0/4022322406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 msgr2=0x557eaa8b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d7fff640 1 -- 172.21.15.19:0/4022322406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eaa5f07e0 con 0x557eaa4e7550 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.320+0000 7f14d77fe640 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.321+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f14c00afc60 0x7f14c00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.321+0000 7f14d57fa640 1 -- 172.21.15.19:0/2760584050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f14cc0deb20 con 0x7f14d8108a20 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.321+0000 7f14d7fff640 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4e7550 0x557eaa8b0b60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f14c800cc00 tx=0x7f14c800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e973ec6ccd819e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.321+0000 7f14be7fc640 1 -- 172.21.15.19:0/4022322406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8005220 con 0x557eaa4e7550 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.321+0000 7f14be7fc640 1 -- 172.21.15.19:0/4022322406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c8004c50 con 0x557eaa4e7550 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.321+0000 7f14be7fc640 1 -- 172.21.15.19:0/4022322406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8010790 con 0x557eaa4e7550 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 -- 172.21.15.19:0/4022322406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4e7550 msgr2=0x557eaa8b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f14c00afc60 0x7f14c00b2170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x557eaa8ad790 tx=0x7f14d002f3c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4e7550 0x557eaa8b0b60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f14c800cc00 tx=0x7f14c800e720 comp rx=0 tx=0).stop 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 -- 172.21.15.19:0/4022322406 shutdown_connections 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4e7550 0x557eaa8b0b60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 --2- 172.21.15.19:0/4022322406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 -- 172.21.15.19:0/4022322406 >> 172.21.15.19:0/4022322406 conn(0x557eaa504560 msgr2=0x557eaa8acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 -- 172.21.15.19:0/4022322406 shutdown_connections 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 -- 172.21.15.19:0/4022322406 wait complete. 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 -- start start 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.322+0000 7f14e55ce740 1 Processor -- start 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8babb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4ef730 0x557eaa8bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14e55ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557eaa8bd970 con 0x557eaa4ef730 2024-04-25T11:03:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14e55ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557eaa7b05d0 con 0x557eaa4e7550 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14e55ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557eaa8bdaf0 con 0x557eaa510410 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14e55ce740 10 client.0 _ll_register_callbacks cb 0x557eaa79e080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4ef730 0x557eaa8bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4ef730 0x557eaa8bb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59278/0 (socket says 172.21.15.19:59278) 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 -- 172.21.15.19:0/2676350274 learned_addr learned my addr 172.21.15.19:0/2676350274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8babb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 -- 172.21.15.19:0/2676350274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 msgr2=0x557eaa8c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 -- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 msgr2=0x557eaa8babb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8babb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 -- 172.21.15.19:0/2676350274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eaa650f70 con 0x557eaa4ef730 2024-04-25T11:03:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8babb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4ef730 0x557eaa8bb0f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0dc850 tx=0x7f14cc0a65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8ec384d9535c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.323+0000 7f14b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14b77fe640 1 -- 172.21.15.19:0/2676350274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc020050 con 0x557eaa4ef730 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14b77fe640 1 -- 172.21.15.19:0/2676350274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f14cc0a2f30 con 0x557eaa4ef730 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c8004c50 con 0x557eaa4ef730 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14b77fe640 1 -- 172.21.15.19:0/2676350274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f14cc0a73a0 con 0x557eaa4ef730 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 10 client.16866 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557eaa4e8170 con 0x557eaa4ef730 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14b77fe640 1 -- 172.21.15.19:0/2676350274 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c8004c50 con 0x557eaa4ef730 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14b77fe640 1 -- 172.21.15.19:0/2676350274 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f14cc0a2610 con 0x557eaa4ef730 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 10 client.16866 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 20 client.16866 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4cf1f0 0x557eaa8ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 4 client.16866 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.324+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eaa824710 con 0x557eaa4cf1f0 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.325+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4cf1f0 0x557eaa8ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.325+0000 7f14b77fe640 10 client.16866 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.325+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4cf1f0 0x557eaa8ccfb0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc03f4ecdf7de826 server_cookie=7b370b47c8b0e172 in_seq=0 out_seq=0 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14b77fe640 1 -- 172.21.15.19:0/2676350274 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557eaa824710 con 0x557eaa4cf1f0 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14b77fe640 10 client.16866 handle_command_reply: tid=0 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 client.16866 shutdown 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4ef730 msgr2=0x557eaa8bb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4ef730 0x557eaa8bb0f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0dc850 tx=0x7f14cc0a65f0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 shutdown_connections 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4cf1f0 0x557eaa8ccfb0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa510410 0x557eaa8c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa4ef730 0x557eaa8bb0f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2676350274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8babb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 >> 172.21.15.19:0/2676350274 conn(0x557eaa504560 msgr2=0x557eaa8b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14b77fe640 0 client.16866 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 shutdown_connections 2024-04-25T11:03:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 1 -- 172.21.15.19:0/2676350274 wait complete. 2024-04-25T11:03:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.326+0000 7f14e55ce740 20 client.16866 trim_cache size 0 max 16384 2024-04-25T11:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 Processor -- start 2024-04-25T11:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 -- start start 2024-04-25T11:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4e7550 0x557eaa8abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ada10 con 0x557eaa510410 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81b0270 con 0x557eaa4ef730 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14e55ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ae060 con 0x557eaa4e7550 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59282/0 (socket says 172.21.15.19:59282) 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 -- 172.21.15.19:0/3776254716 learned_addr learned my addr 172.21.15.19:0/3776254716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14d77fe640 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14d7fff640 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4e7550 0x557eaa8abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 -- 172.21.15.19:0/3776254716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4e7550 msgr2=0x557eaa8abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4e7550 0x557eaa8abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 -- 172.21.15.19:0/3776254716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 msgr2=0x557eaa8b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.345+0000 7f14dccbc640 1 -- 172.21.15.19:0/3776254716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eaa8b1c10 con 0x557eaa510410 2024-04-25T11:03:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.346+0000 7f14dccbc640 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8b48e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f14d0035000 tx=0x7f14d0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db99413c66b242bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.346+0000 7f14bdffb640 1 -- 172.21.15.19:0/3776254716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0003970 con 0x557eaa510410 2024-04-25T11:03:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.346+0000 7f14bdffb640 1 -- 172.21.15.19:0/3776254716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d0036040 con 0x557eaa510410 2024-04-25T11:03:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.346+0000 7f14bdffb640 1 -- 172.21.15.19:0/3776254716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d002f620 con 0x557eaa510410 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.346+0000 7f14e55ce740 1 -- 172.21.15.19:0/3776254716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 msgr2=0x557eaa8b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.346+0000 7f14e55ce740 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8b48e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f14d0035000 tx=0x7f14d0040000 comp rx=0 tx=0).stop 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 -- 172.21.15.19:0/3776254716 shutdown_connections 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8b48e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4ef730 0x557eaa8b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 --2- 172.21.15.19:0/3776254716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4e7550 0x557eaa8abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 -- 172.21.15.19:0/3776254716 >> 172.21.15.19:0/3776254716 conn(0x557eaa504560 msgr2=0x557eaa8ab4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 -- 172.21.15.19:0/3776254716 shutdown_connections 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 -- 172.21.15.19:0/3776254716 wait complete. 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 -- start start 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.347+0000 7f14e55ce740 1 Processor -- start 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4ef730 0x557eaa8c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ae730 con 0x557eaa510410 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81afba0 con 0x557eaa4e7550 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81aee00 con 0x557eaa4ef730 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 10 client.0 _ll_register_callbacks cb 0x7f14cc0a2b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8c6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59286/0 (socket says 172.21.15.19:59286) 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 -- 172.21.15.19:0/2232553776 learned_addr learned my addr 172.21.15.19:0/2232553776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4ef730 0x557eaa8c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 -- 172.21.15.19:0/2232553776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4ef730 msgr2=0x557eaa8c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4ef730 0x557eaa8c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 -- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 msgr2=0x557eaa8c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 -- 172.21.15.19:0/2232553776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eaa8c6e70 con 0x557eaa510410 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14d77fe640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4ef730 0x557eaa8c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14dccbc640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8c6880 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f14d002ade0 tx=0x7f14d0030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0575a3dac9ba6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14b77fe640 1 -- 172.21.15.19:0/2232553776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0056070 con 0x557eaa510410 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d0036040 con 0x557eaa510410 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.348+0000 7f14b77fe640 1 -- 172.21.15.19:0/2232553776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f14d002f580 con 0x557eaa510410 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14b77fe640 1 -- 172.21.15.19:0/2232553776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f14d002f780 con 0x557eaa510410 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 10 client.16872 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557eaa8c1bd0 con 0x557eaa510410 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14b77fe640 1 -- 172.21.15.19:0/2232553776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d0036040 con 0x557eaa510410 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14b77fe640 1 -- 172.21.15.19:0/2232553776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f14d0044040 con 0x557eaa510410 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 10 client.16872 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 20 client.16872 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4a4630 0x557eaa937880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 4 client.16872 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557eaa81e580 con 0x557eaa4a4630 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.349+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4a4630 0x557eaa937880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.350+0000 7f14b77fe640 10 client.16872 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.350+0000 7f14d7fff640 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4a4630 0x557eaa937880 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb3a937f989f7ba server_cookie=8c9a37af8f1542ff in_seq=0 out_seq=0 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14b77fe640 1 -- 172.21.15.19:0/2232553776 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557eaa81e580 con 0x557eaa4a4630 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14b77fe640 10 client.16872 handle_command_reply: tid=0 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 client.16872 shutdown 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 msgr2=0x557eaa8c6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8c6880 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f14d002ade0 tx=0x7f14d0030cf0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 shutdown_connections 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x557eaa4a4630 0x557eaa937880 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557eaa510410 0x557eaa8c6880 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557eaa4ef730 0x557eaa8c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 --2- 172.21.15.19:0/2232553776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eaa4e7550 0x557eaa8c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 >> 172.21.15.19:0/2232553776 conn(0x557eaa504560 msgr2=0x557eaa8b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14b77fe640 0 client.16872 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 shutdown_connections 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 1 -- 172.21.15.19:0/2232553776 wait complete. 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.351+0000 7f14e55ce740 20 client.16872 trim_cache size 0 max 16384 2024-04-25T11:03:09.357 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:03:09.358 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:03:09.358 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "62e700ea-5002-4b7b-8e15-14007b300d93", 2024-04-25T11:03:09.358 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:03:09.358 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f14c00afc60 msgr2=0x7f14c00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f14c00afc60 0x7f14c00b2170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x557eaa8ad790 tx=0x7f14d002f3c0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8108a20 msgr2=0x7f14d819ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8108a20 0x7f14d819ac20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00d770 tx=0x7f14cc00dc30 comp rx=0 tx=0).stop 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 shutdown_connections 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f14c00afc60 0x7f14c00b2170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d8108ec0 0x7f14d819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8108a20 0x7f14d819ac20 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 --2- 172.21.15.19:0/2760584050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d81031d0 0x7f14d819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 >> 172.21.15.19:0/2760584050 conn(0x7f14d8078180 msgr2=0x7f14d8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 shutdown_connections 2024-04-25T11:03:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.352+0000 7f14de71f640 1 -- 172.21.15.19:0/2760584050 wait complete. 2024-04-25T11:03:09.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:03:09.363 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:03:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 -- 172.21.15.19:0/1950331127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 msgr2=0x7f16cc105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 --2- 172.21.15.19:0/1950331127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc105050 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009900 tx=0x7f16bc02a410 comp rx=0 tx=0).stop 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 -- 172.21.15.19:0/1950331127 shutdown_connections 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 --2- 172.21.15.19:0/1950331127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16cc10e090 0x7f16cc110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 --2- 172.21.15.19:0/1950331127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc10b6a0 0x7f16cc10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 --2- 172.21.15.19:0/1950331127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc105050 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 -- 172.21.15.19:0/1950331127 >> 172.21.15.19:0/1950331127 conn(0x7f16cc0fe260 msgr2=0x7f16cc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 -- 172.21.15.19:0/1950331127 shutdown_connections 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.477+0000 7f16d3482640 1 -- 172.21.15.19:0/1950331127 wait complete. 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 Processor -- start 2024-04-25T11:03:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 -- start start 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc10b6a0 0x7f16cc19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16cc10e090 0x7f16cc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc113cf0 con 0x7f16cc104c40 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc113b70 con 0x7f16cc10e090 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d3482640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc113e70 con 0x7f16cc10b6a0 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc19a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59302/0 (socket says 172.21.15.19:59302) 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 -- 172.21.15.19:0/2774932681 learned_addr learned my addr 172.21.15.19:0/2774932681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16cc10e090 0x7f16cc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc10b6a0 0x7f16cc19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 -- 172.21.15.19:0/2774932681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc10b6a0 msgr2=0x7f16cc19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc10b6a0 0x7f16cc19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 -- 172.21.15.19:0/2774932681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16cc10e090 msgr2=0x7f16cc19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16cc10e090 0x7f16cc19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 -- 172.21.15.19:0/2774932681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc19f610 con 0x7f16cc104c40 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16cc10e090 0x7f16cc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc10b6a0 0x7f16cc19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.478+0000 7f16d121e640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc19a7b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005940 tx=0x7f16bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.479+0000 7f16c27fc640 1 -- 172.21.15.19:0/2774932681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc03f030 con 0x7f16cc104c40 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.479+0000 7f16c27fc640 1 -- 172.21.15.19:0/2774932681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16bc003aa0 con 0x7f16cc104c40 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.479+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc19f8a0 con 0x7f16cc104c40 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.479+0000 7f16c27fc640 1 -- 172.21.15.19:0/2774932681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc035070 con 0x7f16cc104c40 2024-04-25T11:03:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.479+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc19fd90 con 0x7f16cc104c40 2024-04-25T11:03:09.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.481+0000 7f16c27fc640 1 -- 172.21.15.19:0/2774932681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f16bc042030 con 0x7f16cc104c40 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.481+0000 7f16da40d740 1 Processor -- start 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.481+0000 7f16da40d740 1 -- start start 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16da40d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d744390 0x561a3dae4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16da40d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dae4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16da40d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d7236b0 0x561a3daf18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16da40d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a3da4ad80 con 0x561a3d744390 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16da40d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a3da048b0 con 0x561a3d71b4d0 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16da40d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561a3da46d40 con 0x561a3d7236b0 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d744390 0x561a3dae4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d744390 0x561a3dae4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59308/0 (socket says 172.21.15.19:59308) 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 -- 172.21.15.19:0/556260350 learned_addr learned my addr 172.21.15.19:0/556260350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d121e640 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dae4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d7236b0 0x561a3daf18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 -- 172.21.15.19:0/556260350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d7236b0 msgr2=0x561a3daf18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d7236b0 0x561a3daf18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 -- 172.21.15.19:0/556260350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 msgr2=0x561a3dae4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dae4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 -- 172.21.15.19:0/556260350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3d9d5a40 con 0x561a3d744390 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d7236b0 0x561a3daf18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16c27fc640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f16b80afc60 0x7f16b80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d744390 0x561a3dae4a60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009fd0 tx=0x7f16c400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafce1a6b4952064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.482+0000 7f16d121e640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f16b80afc60 0x7f16b80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.483+0000 7f16b77fe640 1 -- 172.21.15.19:0/556260350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4015070 con 0x561a3d744390 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.483+0000 7f16c27fc640 1 -- 172.21.15.19:0/2774932681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f16bc0ff810 con 0x7f16cc104c40 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.483+0000 7f16b77fe640 1 -- 172.21.15.19:0/556260350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c4011bb0 con 0x561a3d744390 2024-04-25T11:03:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.483+0000 7f16b77fe640 1 -- 172.21.15.19:0/556260350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4010b20 con 0x561a3d744390 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.483+0000 7f16da40d740 1 -- 172.21.15.19:0/556260350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d744390 msgr2=0x561a3dae4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.483+0000 7f16d121e640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f16b80afc60 0x7f16b80b2170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0fcff0 tx=0x7f16bc105550 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.483+0000 7f16da40d740 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d744390 0x561a3dae4a60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009fd0 tx=0x7f16c400f7f0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 -- 172.21.15.19:0/556260350 shutdown_connections 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d7236b0 0x561a3daf18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dae4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 --2- 172.21.15.19:0/556260350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d744390 0x561a3dae4a60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 -- 172.21.15.19:0/556260350 >> 172.21.15.19:0/556260350 conn(0x561a3d7384e0 msgr2=0x561a3dae1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 -- 172.21.15.19:0/556260350 shutdown_connections 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 -- 172.21.15.19:0/556260350 wait complete. 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 -- start start 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.484+0000 7f16da40d740 1 Processor -- start 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3daef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3daefbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3daf54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a3daf1fe0 con 0x561a3d7236b0 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a3d9d1d10 con 0x561a3d744390 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561a3daf2160 con 0x561a3d71b4d0 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 10 client.0 _ll_register_callbacks cb 0x561a3d9fe210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3daefbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3daefbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59322/0 (socket says 172.21.15.19:59322) 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1058033031 learned_addr learned my addr 172.21.15.19:0/1058033031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3daef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3daf54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1058033031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 msgr2=0x561a3daef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3daef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 msgr2=0x561a3daf54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3daf54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1058033031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3d8267d0 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3daf54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3daefbd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f16c80126f0 tx=0x7f16c8012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c6895f31b214fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1058033031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004460 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1058033031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f16c8010950 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.485+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c4011bb0 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1058033031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f16c8010b50 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 10 client.16884 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561a3d75c3f0 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1058033031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c4011bb0 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1058033031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f16c8019070 con 0x561a3d7236b0 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 10 client.16884 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 20 client.16884 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d702f60 0x561a3db017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 4 client.16884 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a3da58990 con 0x561a3d702f60 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.486+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d702f60 0x561a3db017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.487+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d702f60 0x561a3db017b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=265cb5cb87d459f9 server_cookie=a757e6d65a8af4b3 in_seq=0 out_seq=0 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.487+0000 7f16b4ff9640 10 client.16884 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1058033031 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a3da58990 con 0x561a3d702f60 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16b4ff9640 10 client.16884 handle_command_reply: tid=0 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 client.16884 shutdown 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 msgr2=0x561a3daefbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3daefbd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f16c80126f0 tx=0x7f16c8012bb0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 shutdown_connections 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d702f60 0x561a3db017b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3daf54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3daefbd0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 --2- 172.21.15.19:0/1058033031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3daef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16b4ff9640 0 client.16884 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 >> 172.21.15.19:0/1058033031 conn(0x561a3d7384e0 msgr2=0x561a3dae6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 shutdown_connections 2024-04-25T11:03:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 1 -- 172.21.15.19:0/1058033031 wait complete. 2024-04-25T11:03:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.488+0000 7f16da40d740 20 client.16884 trim_cache size 0 max 16384 2024-04-25T11:03:09.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:09 smithi169 ceph-mon[33049]: pgmap v267: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:03:09.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 Processor -- start 2024-04-25T11:03:09.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 -- start start 2024-04-25T11:03:09.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3dae4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3db06f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3db07470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1adad0 con 0x561a3d7236b0 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1b0330 con 0x561a3d744390 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.506+0000 7f16da40d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1ae120 con 0x561a3d71b4d0 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3db06f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3db06f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59334/0 (socket says 172.21.15.19:59334) 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d1a1f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3dae4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 -- 172.21.15.19:0/3229205452 learned_addr learned my addr 172.21.15.19:0/3229205452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3db07470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 -- 172.21.15.19:0/3229205452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 msgr2=0x561a3dae4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3dae4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 -- 172.21.15.19:0/3229205452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 msgr2=0x561a3db07470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3db07470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 -- 172.21.15.19:0/3229205452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3d71c0f0 con 0x561a3d7236b0 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3db06f30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f16c800a010 tx=0x7f16c8010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2088d3a56f7ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16b6ffd640 1 -- 172.21.15.19:0/3229205452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c80052d0 con 0x561a3d7236b0 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16b6ffd640 1 -- 172.21.15.19:0/3229205452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c801b800 con 0x561a3d7236b0 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.507+0000 7f16b6ffd640 1 -- 172.21.15.19:0/3229205452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8003a00 con 0x561a3d7236b0 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 -- 172.21.15.19:0/3229205452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 msgr2=0x561a3db06f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3db06f30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f16c800a010 tx=0x7f16c8010ae0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 -- 172.21.15.19:0/3229205452 shutdown_connections 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d744390 0x561a3db07470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3db06f30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 --2- 172.21.15.19:0/3229205452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d71b4d0 0x561a3dae4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 -- 172.21.15.19:0/3229205452 >> 172.21.15.19:0/3229205452 conn(0x561a3d7384e0 msgr2=0x561a3dae47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 -- 172.21.15.19:0/3229205452 shutdown_connections 2024-04-25T11:03:09.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 -- 172.21.15.19:0/3229205452 wait complete. 2024-04-25T11:03:09.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 -- start start 2024-04-25T11:03:09.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.508+0000 7f16da40d740 1 Processor -- start 2024-04-25T11:03:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16da40d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dafb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16da40d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3dafb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16da40d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d744390 0x561a3daf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16da40d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1ae7f0 con 0x561a3d7236b0 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16da40d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1afc60 con 0x561a3d71b4d0 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16da40d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1aeec0 con 0x561a3d744390 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16da40d740 10 client.0 _ll_register_callbacks cb 0x7f16c8009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3dafb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3dafb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59338/0 (socket says 172.21.15.19:59338) 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1358972208 learned_addr learned my addr 172.21.15.19:0/1358972208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dafb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1358972208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d744390 msgr2=0x561a3daf6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d744390 0x561a3daf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 msgr2=0x561a3dafb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dafb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 -- 172.21.15.19:0/1358972208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3da0b4b0 con 0x561a3d7236b0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16d0a1d640 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3dafb840 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009fb0 tx=0x7f16c801bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397eb5ad75990aed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.509+0000 7f16b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1358972208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c80052d0 con 0x561a3d7236b0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1358972208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7f16c8005470 con 0x561a3d7236b0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c801b800 con 0x561a3d7236b0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1358972208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f16c8005670 con 0x561a3d7236b0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 10 client.16890 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561a3db69430 con 0x561a3d7236b0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1358972208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c801b800 con 0x561a3d7236b0 2024-04-25T11:03:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1358972208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f16c802a070 con 0x561a3d7236b0 2024-04-25T11:03:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 10 client.16890 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 20 client.16890 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d6d83a0 0x561a3db6baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 4 client.16890 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.510+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a3da52800 con 0x561a3d6d83a0 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.511+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d6d83a0 0x561a3db6baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.511+0000 7f16d1a1f640 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d6d83a0 0x561a3db6baf0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c6a0a80fa42664 server_cookie=ace5358f44ac66df in_seq=0 out_seq=0 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.511+0000 7f16b4ff9640 10 client.16890 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.511+0000 7f16b4ff9640 1 -- 172.21.15.19:0/1358972208 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561a3da52800 con 0x561a3d6d83a0 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.511+0000 7f16b4ff9640 10 client.16890 handle_command_reply: tid=0 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.511+0000 7f16da40d740 1 client.16890 shutdown 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.511+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 msgr2=0x561a3dafb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3dafb840 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009fb0 tx=0x7f16c801bbe0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 shutdown_connections 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561a3d6d83a0 0x561a3db6baf0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a3d744390 0x561a3daf6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16b4ff9640 0 client.16890 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a3d7236b0 0x561a3dafb840 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 --2- 172.21.15.19:0/1358972208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a3d71b4d0 0x561a3dafb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 >> 172.21.15.19:0/1358972208 conn(0x561a3d7384e0 msgr2=0x561a3dae6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 shutdown_connections 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 1 -- 172.21.15.19:0/1358972208 wait complete. 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16da40d740 20 client.16890 trim_cache size 0 max 16384 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:03:09.520 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout: "62e700ea-5002-4b7b-8e15-14007b300d93": { 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "62e700ea-5002-4b7b-8e15-14007b300d93", 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:03:09.521 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:03:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f16b80afc60 msgr2=0x7f16b80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f16b80afc60 0x7f16b80b2170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0fcff0 tx=0x7f16bc105550 comp rx=0 tx=0).stop 2024-04-25T11:03:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 msgr2=0x7f16cc19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc19a7b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005940 tx=0x7f16bc004270 comp rx=0 tx=0).stop 2024-04-25T11:03:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 shutdown_connections 2024-04-25T11:03:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f16b80afc60 0x7f16b80b2170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16cc10e090 0x7f16cc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc10b6a0 0x7f16cc19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 --2- 172.21.15.19:0/2774932681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc104c40 0x7f16cc19a7b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 >> 172.21.15.19:0/2774932681 conn(0x7f16cc0fe260 msgr2=0x7f16cc10e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 shutdown_connections 2024-04-25T11:03:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:09.512+0000 7f16d3482640 1 -- 172.21.15.19:0/2774932681 wait complete. 2024-04-25T11:03:09.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:09 smithi019 ceph-mon[29209]: pgmap v267: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:03:09.524 INFO:tasks.cephfs.filesystem:scrub status for tag:62e700ea-5002-4b7b-8e15-14007b300d93 - {'path': '/', 'tag': '62e700ea-5002-4b7b-8e15-14007b300d93', 'options': 'recursive,force'} 2024-04-25T11:03:09.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:09 smithi112 ceph-mon[32799]: pgmap v267: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:03:10.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:10 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:03:10.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:10 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:10.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:10 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:03:10.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:10 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:10.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:10 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:03:10.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:10 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:10.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:10 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:03:10.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:10 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:10.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:10 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:03:10.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:10 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:10.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:10 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:03:10.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:10 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:11.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:11 smithi112 ceph-mon[32799]: pgmap v268: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:03:11.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:11 smithi169 ceph-mon[33049]: pgmap v268: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:03:11.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:11 smithi019 ceph-mon[29209]: pgmap v268: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:03:12.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:12 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:03:12.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:12 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:03:12.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:12 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:03:13.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:13 smithi169 ceph-mon[33049]: pgmap v269: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 13 KiB/s wr, 4 op/s 2024-04-25T11:03:13.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:13 smithi169 ceph-mon[33049]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:03:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:13 smithi112 ceph-mon[32799]: pgmap v269: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 13 KiB/s wr, 4 op/s 2024-04-25T11:03:13.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:13 smithi112 ceph-mon[32799]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:03:13.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:13 smithi019 ceph-mon[29209]: pgmap v269: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 13 KiB/s wr, 4 op/s 2024-04-25T11:03:13.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:13 smithi019 ceph-mon[29209]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:03:15.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:15 smithi169 ceph-mon[33049]: pgmap v271: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 B/s wr, 2 op/s 2024-04-25T11:03:15.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:15 smithi112 ceph-mon[32799]: pgmap v271: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 B/s wr, 2 op/s 2024-04-25T11:03:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:15 smithi019 ceph-mon[29209]: pgmap v271: 97 pgs: 97 active+clean; 11 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 B/s wr, 2 op/s 2024-04-25T11:03:17.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:17 smithi169 ceph-mon[33049]: pgmap v272: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 229 B/s wr, 4 op/s 2024-04-25T11:03:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:17 smithi112 ceph-mon[32799]: pgmap v272: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 229 B/s wr, 4 op/s 2024-04-25T11:03:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:17 smithi019 ceph-mon[29209]: pgmap v272: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 229 B/s wr, 4 op/s 2024-04-25T11:03:19.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:19 smithi169 ceph-mon[33049]: pgmap v273: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:03:19.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:19 smithi112 ceph-mon[32799]: pgmap v273: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:03:19.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:19 smithi019 ceph-mon[29209]: pgmap v273: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:03:21.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:21 smithi169 ceph-mon[33049]: pgmap v274: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:03:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:21 smithi112 ceph-mon[32799]: pgmap v274: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:03:21.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:21 smithi019 ceph-mon[29209]: pgmap v274: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:03:23.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:23 smithi169 ceph-mon[33049]: pgmap v275: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:03:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:23 smithi112 ceph-mon[32799]: pgmap v275: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:03:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:23 smithi019 ceph-mon[29209]: pgmap v275: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T11:03:25.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:25 smithi169 ceph-mon[33049]: pgmap v276: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:03:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:25 smithi112 ceph-mon[32799]: pgmap v276: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:03:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:25 smithi019 ceph-mon[29209]: pgmap v276: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 89 B/s wr, 1 op/s 2024-04-25T11:03:27.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:27 smithi169 ceph-mon[33049]: pgmap v277: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:03:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:27 smithi112 ceph-mon[32799]: pgmap v277: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:03:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:27 smithi019 ceph-mon[29209]: pgmap v277: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:03:29.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:29 smithi169 ceph-mon[33049]: pgmap v278: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:29.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:29 smithi112 ceph-mon[32799]: pgmap v278: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:29.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:29 smithi019 ceph-mon[29209]: pgmap v278: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:31.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:31 smithi169 ceph-mon[33049]: pgmap v279: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:31.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:31 smithi112 ceph-mon[32799]: pgmap v279: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:31.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:31 smithi019 ceph-mon[29209]: pgmap v279: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:33.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:33 smithi169 ceph-mon[33049]: pgmap v280: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:33.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:33 smithi112 ceph-mon[32799]: pgmap v280: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:33.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:33 smithi019 ceph-mon[29209]: pgmap v280: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:35.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:35 smithi169 ceph-mon[33049]: pgmap v281: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:35.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:35 smithi112 ceph-mon[32799]: pgmap v281: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:35.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:35 smithi019 ceph-mon[29209]: pgmap v281: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:37.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:37 smithi169 ceph-mon[33049]: pgmap v282: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:37.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:37 smithi112 ceph-mon[32799]: pgmap v282: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:37.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:37 smithi019 ceph-mon[29209]: pgmap v282: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:39.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:39 smithi169 ceph-mon[33049]: pgmap v283: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:39.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:03:39.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:03:39.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:39 smithi019 ceph-mon[29209]: pgmap v283: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:39.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:39 smithi112 ceph-mon[32799]: pgmap v283: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:39.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 -- 172.21.15.19:0/2398253660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa88010b680 msgr2=0x7fa88010daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 --2- 172.21.15.19:0/2398253660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa88010b680 0x7fa88010daa0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa874009900 tx=0x7fa87402a480 comp rx=0 tx=0).stop 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 -- 172.21.15.19:0/2398253660 shutdown_connections 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 --2- 172.21.15.19:0/2398253660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa88010e070 0x7fa880110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 --2- 172.21.15.19:0/2398253660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa88010b680 0x7fa88010daa0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 --2- 172.21.15.19:0/2398253660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880104c20 0x7fa880105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 -- 172.21.15.19:0/2398253660 >> 172.21.15.19:0/2398253660 conn(0x7fa880078180 msgr2=0x7fa8800785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 -- 172.21.15.19:0/2398253660 shutdown_connections 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.654+0000 7fa88617a640 1 -- 172.21.15.19:0/2398253660 wait complete. 2024-04-25T11:03:39.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 Processor -- start 2024-04-25T11:03:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 -- start start 2024-04-25T11:03:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880104c20 0x7fa88019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa88010b680 0x7fa88019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88010e070 0x7fa88019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa880113cd0 con 0x7fa880104c20 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa880113b50 con 0x7fa88010b680 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa88617a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa880113e50 con 0x7fa88010e070 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880104c20 0x7fa88019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880104c20 0x7fa88019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48542/0 (socket says 172.21.15.19:48542) 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.655+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2058378175 learned_addr learned my addr 172.21.15.19:0/2058378175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88010e070 0x7fa88019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87effd640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa88010b680 0x7fa88019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2058378175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88010e070 msgr2=0x7fa88019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88010e070 0x7fa88019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2058378175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa88010b680 msgr2=0x7fa88019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa88010b680 0x7fa88019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2058378175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88019f5b0 con 0x7fa880104c20 2024-04-25T11:03:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88010e070 0x7fa88019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880104c20 0x7fa88019a750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c970 tx=0x7fa87000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87cff9640 1 -- 172.21.15.19:0/2058378175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870007bb0 con 0x7fa880104c20 2024-04-25T11:03:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87cff9640 1 -- 172.21.15.19:0/2058378175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa870007d50 con 0x7fa880104c20 2024-04-25T11:03:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88019f8a0 con 0x7fa880104c20 2024-04-25T11:03:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa87cff9640 1 -- 172.21.15.19:0/2058378175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000f450 con 0x7fa880104c20 2024-04-25T11:03:39.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.656+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88019fd70 con 0x7fa880104c20 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa87cff9640 1 -- 172.21.15.19:0/2058378175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa870004f20 con 0x7fa880104c20 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 Processor -- start 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 -- start start 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b014a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac4b4d0 0x55601b014fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55601af7ad80 con 0x55601ac4b4d0 2024-04-25T11:03:39.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55601af348b0 con 0x55601ac536b0 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa88d022740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55601af76d40 con 0x55601ac74390 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.659+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac4b4d0 0x55601b014fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac4b4d0 0x55601b014fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48548/0 (socket says 172.21.15.19:48548) 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2827305683 learned_addr learned my addr 172.21.15.19:0/2827305683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b014a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87effd640 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87cff9640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8680afc60 0x7fa8680b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2827305683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 msgr2=0x55601b014a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b014a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2827305683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 msgr2=0x55601b0218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0218f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 -- 172.21.15.19:0/2827305683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601af05a40 con 0x55601ac4b4d0 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87effd640 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:39.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8680afc60 0x7fa8680b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87cff9640 1 -- 172.21.15.19:0/2058378175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7fa8700deea0 con 0x7fa880104c20 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac4b4d0 0x55601b014fa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa8700dc560 tx=0x7fa87006d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44f7703a8e1e73e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8680afc60 0x7fa8680b2170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55601b011bd0 tx=0x7fa878004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa865ffb640 1 -- 172.21.15.19:0/2827305683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87001d070 con 0x55601ac4b4d0 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa865ffb640 1 -- 172.21.15.19:0/2827305683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8700055f0 con 0x55601ac4b4d0 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.660+0000 7fa865ffb640 1 -- 172.21.15.19:0/2827305683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8700a6b50 con 0x55601ac4b4d0 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 -- 172.21.15.19:0/2827305683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac4b4d0 msgr2=0x55601b014fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac4b4d0 0x55601b014fa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa8700dc560 tx=0x7fa87006d460 comp rx=0 tx=0).stop 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 -- 172.21.15.19:0/2827305683 shutdown_connections 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac4b4d0 0x55601b014fa0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 --2- 172.21.15.19:0/2827305683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b014a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 -- 172.21.15.19:0/2827305683 >> 172.21.15.19:0/2827305683 conn(0x55601ac684e0 msgr2=0x55601b011210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:39.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 -- 172.21.15.19:0/2827305683 shutdown_connections 2024-04-25T11:03:39.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.661+0000 7fa88d022740 1 -- 172.21.15.19:0/2827305683 wait complete. 2024-04-25T11:03:39.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 -- start start 2024-04-25T11:03:39.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 Processor -- start 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac536b0 0x55601b01fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b0253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55601b021f80 con 0x55601ac536b0 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55601af01d10 con 0x55601ac4b4d0 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.662+0000 7fa88d022740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55601b022100 con 0x55601ac74390 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa88d022740 10 client.0 _ll_register_callbacks cb 0x55601af2e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac536b0 0x55601b01fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac536b0 0x55601b01fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48554/0 (socket says 172.21.15.19:48554) 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 -- 172.21.15.19:0/2807457792 learned_addr learned my addr 172.21.15.19:0/2807457792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b0253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 -- 172.21.15.19:0/2807457792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 msgr2=0x55601b0253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b0253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 -- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 msgr2=0x55601b01f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 -- 172.21.15.19:0/2807457792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601ad567d0 con 0x55601ac536b0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b0253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa87effd640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac536b0 0x55601b01fb30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa874033000 tx=0x7fa8740317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c6a4bbdbbb4f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa85effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa85effd640 1 -- 172.21.15.19:0/2807457792 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874035030 con 0x55601ac536b0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55601af88990 con 0x55601ac536b0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.663+0000 7fa85effd640 1 -- 172.21.15.19:0/2807457792 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7fa874004460 con 0x55601ac536b0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa85effd640 1 -- 172.21.15.19:0/2807457792 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa874005030 con 0x55601ac536b0 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 10 client.16902 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:39.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55601ac8c3f0 con 0x55601ac536b0 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa85effd640 1 -- 172.21.15.19:0/2807457792 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55601af88990 con 0x55601ac536b0 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa85effd640 1 -- 172.21.15.19:0/2807457792 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa8740035f0 con 0x55601ac536b0 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 10 client.16902 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 20 client.16902 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac32f60 0x55601b031650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 4 client.16902 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55601af82800 con 0x55601ac32f60 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.664+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac32f60 0x55601b031650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.665+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac32f60 0x55601b031650 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a557414b71be1a server_cookie=3d022ef440a9720e in_seq=0 out_seq=0 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.665+0000 7fa85effd640 10 client.16902 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa85effd640 1 -- 172.21.15.19:0/2807457792 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55601af82800 con 0x55601ac32f60 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa85effd640 10 client.16902 handle_command_reply: tid=0 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 client.16902 shutdown 2024-04-25T11:03:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac536b0 msgr2=0x55601b01fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac536b0 0x55601b01fb30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa874033000 tx=0x7fa8740317c0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 shutdown_connections 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac32f60 0x55601b031650 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac74390 0x55601b0253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac536b0 0x55601b01fb30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa85effd640 0 client.16902 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 --2- 172.21.15.19:0/2807457792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 >> 172.21.15.19:0/2807457792 conn(0x55601ac684e0 msgr2=0x55601b016780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 shutdown_connections 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 1 -- 172.21.15.19:0/2807457792 wait complete. 2024-04-25T11:03:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.666+0000 7fa88d022740 20 client.16902 trim_cache size 0 max 16384 2024-04-25T11:03:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 Processor -- start 2024-04-25T11:03:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 -- start start 2024-04-25T11:03:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac536b0 0x55601b0253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b025900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801ad970 con 0x55601ac74390 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801b01d0 con 0x55601ac4b4d0 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa88d022740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801adfc0 con 0x55601ac536b0 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b025900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b025900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48564/0 (socket says 172.21.15.19:48564) 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87ffff640 1 -- 172.21.15.19:0/2590510839 learned_addr learned my addr 172.21.15.19:0/2590510839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87effd640 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac536b0 0x55601b0253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87ffff640 1 -- 172.21.15.19:0/2590510839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac536b0 msgr2=0x55601b0253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac536b0 0x55601b0253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87ffff640 1 -- 172.21.15.19:0/2590510839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 msgr2=0x55601b01e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.685+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.686+0000 7fa87ffff640 1 -- 172.21.15.19:0/2590510839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601ac4c0f0 con 0x55601ac74390 2024-04-25T11:03:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.686+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.686+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b025900 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa878020860 tx=0x7fa878015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d4a2f054d8820a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.686+0000 7fa8657fa640 1 -- 172.21.15.19:0/2590510839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878020230 con 0x55601ac74390 2024-04-25T11:03:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.686+0000 7fa8657fa640 1 -- 172.21.15.19:0/2590510839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa878017070 con 0x55601ac74390 2024-04-25T11:03:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.686+0000 7fa8657fa640 1 -- 172.21.15.19:0/2590510839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87801fd50 con 0x55601ac74390 2024-04-25T11:03:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 -- 172.21.15.19:0/2590510839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 msgr2=0x55601b025900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b025900 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa878020860 tx=0x7fa878015000 comp rx=0 tx=0).stop 2024-04-25T11:03:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 -- 172.21.15.19:0/2590510839 shutdown_connections 2024-04-25T11:03:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b025900 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac536b0 0x55601b0253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 --2- 172.21.15.19:0/2590510839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac4b4d0 0x55601b01e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 -- 172.21.15.19:0/2590510839 >> 172.21.15.19:0/2590510839 conn(0x55601ac684e0 msgr2=0x55601b01dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 -- 172.21.15.19:0/2590510839 shutdown_connections 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 -- 172.21.15.19:0/2590510839 wait complete. 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 -- start start 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.687+0000 7fa88d022740 1 Processor -- start 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa88d022740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac4b4d0 0x55601b026bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa88d022740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa88d022740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b0979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa88d022740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801ae690 con 0x55601ac74390 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa88d022740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801afb00 con 0x55601ac536b0 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa88d022740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801aed60 con 0x55601ac4b4d0 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa88d022740 10 client.0 _ll_register_callbacks cb 0x7fa874009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b0979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b0979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48574/0 (socket says 172.21.15.19:48574) 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 -- 172.21.15.19:0/2426440835 learned_addr learned my addr 172.21.15.19:0/2426440835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac4b4d0 0x55601b026bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87effd640 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 -- 172.21.15.19:0/2426440835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac4b4d0 msgr2=0x55601b026bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac4b4d0 0x55601b026bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 -- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 msgr2=0x55601b0270f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0270f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 -- 172.21.15.19:0/2426440835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601af3b4b0 con 0x55601ac74390 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa87ffff640 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b0979b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa87801f9f0 tx=0x7fa87801f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b16277c8a9d731b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.688+0000 7fa85effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa85effd640 1 -- 172.21.15.19:0/2426440835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878039060 con 0x55601ac74390 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa878017070 con 0x55601ac74390 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa85effd640 1 -- 172.21.15.19:0/2426440835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7fa878013980 con 0x55601ac74390 2024-04-25T11:03:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa85effd640 1 -- 172.21.15.19:0/2426440835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa878013b80 con 0x55601ac74390 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 10 client.16908 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55601b098e60 con 0x55601ac74390 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa85effd640 1 -- 172.21.15.19:0/2426440835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa878017070 con 0x55601ac74390 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa85effd640 1 -- 172.21.15.19:0/2426440835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa87800f040 con 0x55601ac74390 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 10 client.16908 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 20 client.16908 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac083a0 0x55601b09b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 4 client.16908 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.689+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55601aefd4a0 con 0x55601ac083a0 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac083a0 0x55601b09b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa85effd640 10 client.16908 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa87f7fe640 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac083a0 0x55601b09b5f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae7ad6454b2e9cd server_cookie=7dfe6ecf9e42419c in_seq=0 out_seq=0 2024-04-25T11:03:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa85effd640 1 -- 172.21.15.19:0/2426440835 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55601aefd4a0 con 0x55601ac083a0 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa85effd640 10 client.16908 handle_command_reply: tid=0 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 client.16908 shutdown 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 msgr2=0x55601b0979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b0979b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa87801f9f0 tx=0x7fa87801f730 comp rx=0 tx=0).stop 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 shutdown_connections 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55601ac083a0 0x55601b09b5f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55601ac74390 0x55601b0979b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55601ac536b0 0x55601b0270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa85effd640 0 client.16908 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 --2- 172.21.15.19:0/2426440835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55601ac4b4d0 0x55601b026bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.690+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 >> 172.21.15.19:0/2426440835 conn(0x55601ac684e0 msgr2=0x55601b020ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 shutdown_connections 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88d022740 1 -- 172.21.15.19:0/2426440835 wait complete. 2024-04-25T11:03:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88d022740 20 client.16908 trim_cache size 0 max 16384 2024-04-25T11:03:39.697 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:03:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:03:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:03:39.697 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8680afc60 msgr2=0x7fa8680b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8680afc60 0x7fa8680b2170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55601b011bd0 tx=0x7fa878004a40 comp rx=0 tx=0).stop 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880104c20 msgr2=0x7fa88019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880104c20 0x7fa88019a750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c970 tx=0x7fa87000ce30 comp rx=0 tx=0).stop 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 shutdown_connections 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fa8680afc60 0x7fa8680b2170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88010e070 0x7fa88019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa88010b680 0x7fa88019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 --2- 172.21.15.19:0/2058378175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880104c20 0x7fa88019a750 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 >> 172.21.15.19:0/2058378175 conn(0x7fa880078180 msgr2=0x7fa8801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:39.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 shutdown_connections 2024-04-25T11:03:39.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.691+0000 7fa88617a640 1 -- 172.21.15.19:0/2058378175 wait complete. 2024-04-25T11:03:39.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:03:39.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:03:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 -- 172.21.15.19:0/3171504591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c102b00 msgr2=0x7f185c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 --2- 172.21.15.19:0/3171504591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c102b00 0x7f185c102f10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802a410 comp rx=0 tx=0).stop 2024-04-25T11:03:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 -- 172.21.15.19:0/3171504591 shutdown_connections 2024-04-25T11:03:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 --2- 172.21.15.19:0/3171504591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f185c10d390 0x7f185c10f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 --2- 172.21.15.19:0/3171504591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c102b00 0x7f185c102f10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 --2- 172.21.15.19:0/3171504591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f185c108a30 0x7f185c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 -- 172.21.15.19:0/3171504591 >> 172.21.15.19:0/3171504591 conn(0x7f185c0fe080 msgr2=0x7f185c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 -- 172.21.15.19:0/3171504591 shutdown_connections 2024-04-25T11:03:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 -- 172.21.15.19:0/3171504591 wait complete. 2024-04-25T11:03:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 Processor -- start 2024-04-25T11:03:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.856+0000 7f1863053640 1 -- start start 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1863053640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f185c102b00 0x7f185c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1863053640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f185c108a30 0x7f185c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1863053640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c10d390 0x7f185c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1863053640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f185c113d60 con 0x7f185c10d390 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1863053640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f185c113be0 con 0x7f185c102b00 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1863053640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f185c113ee0 con 0x7f185c108a30 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c10d390 0x7f185c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c10d390 0x7f185c19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48596/0 (socket says 172.21.15.19:48596) 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 -- 172.21.15.19:0/3800034287 learned_addr learned my addr 172.21.15.19:0/3800034287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1860def640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f185c102b00 0x7f185c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 -- 172.21.15.19:0/3800034287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f185c108a30 msgr2=0x7f185c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f185c108a30 0x7f185c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 -- 172.21.15.19:0/3800034287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f185c102b00 msgr2=0x7f185c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f185c102b00 0x7f185c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 -- 172.21.15.19:0/3800034287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c19f560 con 0x7f185c10d390 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f18615f0640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c10d390 0x7f185c19ef20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1854009fd0 tx=0x7f185400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1859ffb640 1 -- 172.21.15.19:0/3800034287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854017070 con 0x7f185c10d390 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1859ffb640 1 -- 172.21.15.19:0/3800034287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1854003e20 con 0x7f185c10d390 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c19f850 con 0x7f185c10d390 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.857+0000 7f1859ffb640 1 -- 172.21.15.19:0/3800034287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854010460 con 0x7f185c10d390 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.858+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c19fd50 con 0x7f185c10d390 2024-04-25T11:03:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.859+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c06a190 con 0x7f185c10d390 2024-04-25T11:03:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.860+0000 7f1859ffb640 1 -- 172.21.15.19:0/3800034287 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1854003970 con 0x7f185c10d390 2024-04-25T11:03:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.861+0000 7f1859ffb640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f184c0afc60 0x7f184c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.861+0000 7f1859ffb640 1 -- 172.21.15.19:0/3800034287 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f18540deec0 con 0x7f185c10d390 2024-04-25T11:03:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.861+0000 7f1860def640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f184c0afc60 0x7f184c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.861+0000 7f1860def640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f184c0afc60 0x7f184c0b2170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f185c103a30 tx=0x7f1850005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:39.866+0000 7f1859ffb640 1 -- 172.21.15.19:0/3800034287 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f185c19fd50 con 0x7f185c10d390 2024-04-25T11:03:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.094+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f185c1a7380 con 0x7f185c10d390 2024-04-25T11:03:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.095+0000 7f1859ffb640 1 -- 172.21.15.19:0/3800034287 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f18540a7760 con 0x7f185c10d390 2024-04-25T11:03:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:03:40.095 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:03:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:03:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f184c0afc60 msgr2=0x7f184c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f184c0afc60 0x7f184c0b2170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f185c103a30 tx=0x7f1850005f90 comp rx=0 tx=0).stop 2024-04-25T11:03:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c10d390 msgr2=0x7f185c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c10d390 0x7f185c19ef20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1854009fd0 tx=0x7f185400b260 comp rx=0 tx=0).stop 2024-04-25T11:03:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 shutdown_connections 2024-04-25T11:03:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f184c0afc60 0x7f184c0b2170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f185c10d390 0x7f185c19ef20 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f185c108a30 0x7f185c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 --2- 172.21.15.19:0/3800034287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f185c102b00 0x7f185c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 >> 172.21.15.19:0/3800034287 conn(0x7f185c0fe080 msgr2=0x7f185c10e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 shutdown_connections 2024-04-25T11:03:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.097+0000 7f1863053640 1 -- 172.21.15.19:0/3800034287 wait complete. 2024-04-25T11:03:40.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 -- 172.21.15.19:0/286674109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a4102b60 msgr2=0x7f02a4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 --2- 172.21.15.19:0/286674109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a4102b60 0x7f02a4102f70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0298009900 tx=0x7f029802a410 comp rx=0 tx=0).stop 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 -- 172.21.15.19:0/286674109 shutdown_connections 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 --2- 172.21.15.19:0/286674109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a410d3a0 0x7f02a410f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 --2- 172.21.15.19:0/286674109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a4102b60 0x7f02a4102f70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 --2- 172.21.15.19:0/286674109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a4108a40 0x7f02a4102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 -- 172.21.15.19:0/286674109 >> 172.21.15.19:0/286674109 conn(0x7f02a40fe260 msgr2=0x7f02a4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 -- 172.21.15.19:0/286674109 shutdown_connections 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.223+0000 7f02a971f640 1 -- 172.21.15.19:0/286674109 wait complete. 2024-04-25T11:03:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 Processor -- start 2024-04-25T11:03:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 -- start start 2024-04-25T11:03:40.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a4102b60 0x7f02a41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a4108a40 0x7f02a4199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a410d3a0 0x7f02a419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4112c00 con 0x7f02a410d3a0 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4112a80 con 0x7f02a4108a40 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a971f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4112d80 con 0x7f02a4102b60 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a410d3a0 0x7f02a419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a410d3a0 0x7f02a419dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48602/0 (socket says 172.21.15.19:48602) 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a37fe640 1 -- 172.21.15.19:0/4117680861 learned_addr learned my addr 172.21.15.19:0/4117680861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.224+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a4102b60 0x7f02a41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a27fc640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a4108a40 0x7f02a4199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a37fe640 1 -- 172.21.15.19:0/4117680861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a4102b60 msgr2=0x7f02a41996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a37fe640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a4102b60 0x7f02a41996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a37fe640 1 -- 172.21.15.19:0/4117680861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a4108a40 msgr2=0x7f02a4199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a37fe640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a4108a40 0x7f02a4199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a37fe640 1 -- 172.21.15.19:0/4117680861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a419e520 con 0x7f02a410d3a0 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a37fe640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a410d3a0 0x7f02a419dee0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f029c00eb10 tx=0x7f029c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f028bfff640 1 -- 172.21.15.19:0/4117680861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c004b20 con 0x7f02a410d3a0 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f028bfff640 1 -- 172.21.15.19:0/4117680861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f029c004cc0 con 0x7f02a410d3a0 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a419e810 con 0x7f02a410d3a0 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f028bfff640 1 -- 172.21.15.19:0/4117680861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c010670 con 0x7f02a410d3a0 2024-04-25T11:03:40.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.225+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41a6190 con 0x7f02a410d3a0 2024-04-25T11:03:40.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 Processor -- start 2024-04-25T11:03:40.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f028bfff640 1 -- 172.21.15.19:0/4117680861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f029c010810 con 0x7f02a410d3a0 2024-04-25T11:03:40.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 -- start start 2024-04-25T11:03:40.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fc01390 0x55575ffa4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbd84d0 0x55575ffa4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffb18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55575ff07d80 con 0x55575fbe06b0 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55575fec1a20 con 0x55575fbd84d0 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02b05d4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55575ff03d40 con 0x55575fc01390 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffb18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffb18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48604/0 (socket says 172.21.15.19:48604) 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02a27fc640 1 -- 172.21.15.19:0/565767052 learned_addr learned my addr 172.21.15.19:0/565767052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f028bfff640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f028c0afc60 0x7f028c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.228+0000 7f02a27fc640 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fc01390 0x55575ffa4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbd84d0 0x55575ffa4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a37fe640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f028c0afc60 0x7f028c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f028bfff640 1 -- 172.21.15.19:0/4117680861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f029c0df630 con 0x7f02a410d3a0 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a27fc640 1 -- 172.21.15.19:0/565767052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fc01390 msgr2=0x55575ffa4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a27fc640 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fc01390 0x55575ffa4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a27fc640 1 -- 172.21.15.19:0/565767052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbd84d0 msgr2=0x55575ffa4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a27fc640 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbd84d0 0x55575ffa4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a27fc640 1 -- 172.21.15.19:0/565767052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575feccc30 con 0x55575fbe06b0 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a27fc640 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffb18b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f02980057c0 tx=0x7f02980058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3313bc13dd2b38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02a37fe640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f028c0afc60 0x7f028c0b2170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f029c0dd380 tx=0x7f029c014000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02897fa640 1 -- 172.21.15.19:0/565767052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298035040 con 0x55575fbe06b0 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02897fa640 1 -- 172.21.15.19:0/565767052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0298031d10 con 0x55575fbe06b0 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.229+0000 7f02897fa640 1 -- 172.21.15.19:0/565767052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02980035f0 con 0x55575fbe06b0 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 -- 172.21.15.19:0/565767052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 msgr2=0x55575ffb18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffb18b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f02980057c0 tx=0x7f02980058b0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 -- 172.21.15.19:0/565767052 shutdown_connections 2024-04-25T11:03:40.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffb18b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbd84d0 0x55575ffa4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 --2- 172.21.15.19:0/565767052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fc01390 0x55575ffa4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 -- 172.21.15.19:0/565767052 >> 172.21.15.19:0/565767052 conn(0x55575fbf54e0 msgr2=0x55575ffa11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 -- 172.21.15.19:0/565767052 shutdown_connections 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.230+0000 7f02b05d4740 1 -- 172.21.15.19:0/565767052 wait complete. 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 -- start start 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 Processor -- start 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbd84d0 0x55575ffaf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbe06b0 0x55575ffaf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x55575ffbb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55575ffb1f40 con 0x55575fbd84d0 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55575fe8edc0 con 0x55575fc01390 2024-04-25T11:03:40.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55575ffb20c0 con 0x55575fbe06b0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02b05d4740 10 client.0 _ll_register_callbacks cb 0x55575feaa600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbd84d0 0x55575ffaf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbd84d0 0x55575ffaf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48608/0 (socket says 172.21.15.19:48608) 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 -- 172.21.15.19:0/220296341 learned_addr learned my addr 172.21.15.19:0/220296341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbe06b0 0x55575ffaf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x55575ffbb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 -- 172.21.15.19:0/220296341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbe06b0 msgr2=0x55575ffaf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbe06b0 0x55575ffaf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 -- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 msgr2=0x55575ffbb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x55575ffbb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.231+0000 7f02a27fc640 1 -- 172.21.15.19:0/220296341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575fc193f0 con 0x55575fbd84d0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02a27fc640 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbd84d0 0x55575ffaf440 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0298001eb0 tx=0x7f0298032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce2c42923f0bf70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f0286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:40.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f0286ff5640 1 -- 172.21.15.19:0/220296341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298051060 con 0x55575fbd84d0 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f0286ff5640 1 -- 172.21.15.19:0/220296341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f029803d690 con 0x55575fbd84d0 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0298031d10 con 0x55575fbd84d0 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f0286ff5640 1 -- 172.21.15.19:0/220296341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0298035050 con 0x55575fbd84d0 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 10 client.16926 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55575fce37d0 con 0x55575fbd84d0 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f0286ff5640 1 -- 172.21.15.19:0/220296341 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0298031d10 con 0x55575fbd84d0 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f0286ff5640 1 -- 172.21.15.19:0/220296341 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f029803a040 con 0x55575fbd84d0 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 10 client.16926 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 20 client.16926 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fbbff60 0x55575ffc1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 4 client.16926 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:40.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.232+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55575ff15990 con 0x55575fbbff60 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.233+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fbbff60 0x55575ffc1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.233+0000 7f0286ff5640 10 client.16926 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.233+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fbbff60 0x55575ffc1720 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232dfab7b29706ed server_cookie=86ce7d6ea1e22343 in_seq=0 out_seq=0 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f0286ff5640 1 -- 172.21.15.19:0/220296341 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55575ff15990 con 0x55575fbbff60 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f0286ff5640 10 client.16926 handle_command_reply: tid=0 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 client.16926 shutdown 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbd84d0 msgr2=0x55575ffaf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbd84d0 0x55575ffaf440 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0298001eb0 tx=0x7f0298032e00 comp rx=0 tx=0).stop 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 shutdown_connections 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fbbff60 0x55575ffc1720 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x55575ffbb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbe06b0 0x55575ffaf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f0286ff5640 0 client.16926 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:40.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 --2- 172.21.15.19:0/220296341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbd84d0 0x55575ffaf440 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 >> 172.21.15.19:0/220296341 conn(0x55575fbf54e0 msgr2=0x55575ffa6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:40.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 shutdown_connections 2024-04-25T11:03:40.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 1 -- 172.21.15.19:0/220296341 wait complete. 2024-04-25T11:03:40.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.234+0000 7f02b05d4740 20 client.16926 trim_cache size 0 max 16384 2024-04-25T11:03:40.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 Processor -- start 2024-04-25T11:03:40.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 -- start start 2024-04-25T11:03:40.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffbb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbe06b0 0x55575ffae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fc01390 0x55575ffa9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ab710 con 0x55575fc01390 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41adf70 con 0x55575fbe06b0 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02b05d4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41abd60 con 0x55575fbd84d0 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fc01390 0x55575ffa9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fc01390 0x55575ffa9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48616/0 (socket says 172.21.15.19:48616) 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.248+0000 7f02a27fc640 1 -- 172.21.15.19:0/4121545138 learned_addr learned my addr 172.21.15.19:0/4121545138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbe06b0 0x55575ffae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a27fc640 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffbb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a27fc640 1 -- 172.21.15.19:0/4121545138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 msgr2=0x55575ffbb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a27fc640 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffbb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a27fc640 1 -- 172.21.15.19:0/4121545138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbe06b0 msgr2=0x55575ffae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a27fc640 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbe06b0 0x55575ffae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a27fc640 1 -- 172.21.15.19:0/4121545138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575ffad780 con 0x55575fc01390 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f02a27fc640 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fc01390 0x55575ffa9640 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x55575ff03e50 tx=0x7f0298031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a4baef8b4744b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f0288ff9640 1 -- 172.21.15.19:0/4121545138 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298051070 con 0x55575fc01390 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f0288ff9640 1 -- 172.21.15.19:0/4121545138 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0298031ca0 con 0x55575fc01390 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.249+0000 7f0288ff9640 1 -- 172.21.15.19:0/4121545138 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029803da50 con 0x55575fc01390 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 -- 172.21.15.19:0/4121545138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fc01390 msgr2=0x55575ffa9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fc01390 0x55575ffa9640 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x55575ff03e50 tx=0x7f0298031400 comp rx=0 tx=0).stop 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 -- 172.21.15.19:0/4121545138 shutdown_connections 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fc01390 0x55575ffa9640 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fbe06b0 0x55575ffae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 --2- 172.21.15.19:0/4121545138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffbb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 -- 172.21.15.19:0/4121545138 >> 172.21.15.19:0/4121545138 conn(0x55575fbf54e0 msgr2=0x55575ff9d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 -- 172.21.15.19:0/4121545138 shutdown_connections 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 -- 172.21.15.19:0/4121545138 wait complete. 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 -- start start 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.250+0000 7f02b05d4740 1 Processor -- start 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffc1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffc2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x555760027840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ac430 con 0x55575fbe06b0 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ad8a0 con 0x55575fc01390 2024-04-25T11:03:40.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41acb00 con 0x55575fbd84d0 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 10 client.0 _ll_register_callbacks cb 0x7f02940097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffc2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffc2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48618/0 (socket says 172.21.15.19:48618) 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 -- 172.21.15.19:0/1300908778 learned_addr learned my addr 172.21.15.19:0/1300908778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a27fc640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x555760027840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 -- 172.21.15.19:0/1300908778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 msgr2=0x55575ffc1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a27fc640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffc1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffc1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 -- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 msgr2=0x555760027840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x555760027840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 -- 172.21.15.19:0/1300908778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555760027db0 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a27fc640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffc1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02a2ffd640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffc2070 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f02940109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b617b87f46475b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f0286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f0286ff5640 1 -- 172.21.15.19:0/1300908778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294004230 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f0286ff5640 1 -- 172.21.15.19:0/1300908778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f02940043d0 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0298031ca0 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f0286ff5640 1 -- 172.21.15.19:0/1300908778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f029401e780 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 10 client.16932 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.251+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555760029070 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f0286ff5640 1 -- 172.21.15.19:0/1300908778 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0298031ca0 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f0286ff5640 1 -- 172.21.15.19:0/1300908778 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f02940200f0 con 0x55575fbe06b0 2024-04-25T11:03:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02b05d4740 10 client.16932 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02b05d4740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02b05d4740 20 client.16932 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02b05d4740 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fb953a0 0x55576002b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02b05d4740 4 client.16932 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55575ff0f800 con 0x55575fb953a0 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02a27fc640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fb953a0 0x55576002b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f0286ff5640 10 client.16932 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.252+0000 7f02a27fc640 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fb953a0 0x55576002b7c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1dd6a6ff77b7c22 server_cookie=632c40d64ff9edd4 in_seq=0 out_seq=0 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f0286ff5640 1 -- 172.21.15.19:0/1300908778 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55575ff0f800 con 0x55575fb953a0 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f0286ff5640 10 client.16932 handle_command_reply: tid=0 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 client.16932 shutdown 2024-04-25T11:03:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 msgr2=0x55575ffc2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffc2070 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f02940109b0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 shutdown_connections 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55575fb953a0 0x55576002b7c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55575fc01390 0x555760027840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55575fbe06b0 0x55575ffc2070 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f0286ff5640 0 client.16932 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 --2- 172.21.15.19:0/1300908778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55575fbd84d0 0x55575ffc1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 >> 172.21.15.19:0/1300908778 conn(0x55575fbf54e0 msgr2=0x55575ffb08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 shutdown_connections 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 1 -- 172.21.15.19:0/1300908778 wait complete. 2024-04-25T11:03:40.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02b05d4740 20 client.16932 trim_cache size 0 max 16384 2024-04-25T11:03:40.262 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:03:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f028c0afc60 msgr2=0x7f028c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02a971f640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f028c0afc60 0x7f028c0b2170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f029c0dd380 tx=0x7f029c014000 comp rx=0 tx=0).stop 2024-04-25T11:03:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a410d3a0 msgr2=0x7f02a419dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.253+0000 7f02a971f640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a410d3a0 0x7f02a419dee0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f029c00eb10 tx=0x7f029c00c2d0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 shutdown_connections 2024-04-25T11:03:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f028c0afc60 0x7f028c0b2170 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a410d3a0 0x7f02a419dee0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a4108a40 0x7f02a4199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 --2- 172.21.15.19:0/4117680861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a4102b60 0x7f02a41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:40.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 >> 172.21.15.19:0/4117680861 conn(0x7f02a40fe260 msgr2=0x7f02a41040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:40.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 shutdown_connections 2024-04-25T11:03:40.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:40.254+0000 7f02a971f640 1 -- 172.21.15.19:0/4117680861 wait complete. 2024-04-25T11:03:40.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:40 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3800034287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:03:40.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:40 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3800034287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:03:40.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:40 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3800034287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:03:41.266 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:03:41.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:03:41.267 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1103825710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec104260 msgr2=0x7f8bec110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1103825710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec104260 0x7f8bec110630 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009ca0 tx=0x7f8be402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1103825710 shutdown_connections 2024-04-25T11:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1103825710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec104260 0x7f8bec110630 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1103825710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bec109fb0 0x7f8bec103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1103825710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bec109b10 0x7f8bec1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1103825710 >> 172.21.15.19:0/1103825710 conn(0x7f8bec0ff230 msgr2=0x7f8bec101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.397+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1103825710 shutdown_connections 2024-04-25T11:03:41.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.398+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1103825710 wait complete. 2024-04-25T11:03:41.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.398+0000 7f8bf15ce640 1 Processor -- start 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.398+0000 7f8bf15ce640 1 -- start start 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bf15ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bec104260 0x7f8bec19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bf15ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec109b10 0x7f8bec19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bf15ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bec109fb0 0x7f8bec19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bf15ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec113d20 con 0x7f8bec109b10 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bf15ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec113ba0 con 0x7f8bec104260 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bf15ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec113ea0 con 0x7f8bec109fb0 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec109b10 0x7f8bec19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec109b10 0x7f8bec19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48634/0 (socket says 172.21.15.19:48634) 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 -- 172.21.15.19:0/1445765033 learned_addr learned my addr 172.21.15.19:0/1445765033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8beaffd640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bec104260 0x7f8bec19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bec109fb0 0x7f8bec19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 -- 172.21.15.19:0/1445765033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bec109fb0 msgr2=0x7f8bec19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bec109fb0 0x7f8bec19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 -- 172.21.15.19:0/1445765033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bec104260 msgr2=0x7f8bec19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bec104260 0x7f8bec19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 -- 172.21.15.19:0/1445765033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec19f540 con 0x7f8bec109b10 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.399+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec109b10 0x7f8bec19ac40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8be0005d00 tx=0x7f8be000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.400+0000 7f8bd3fff640 1 -- 172.21.15.19:0/1445765033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0017070 con 0x7f8bec109b10 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.400+0000 7f8bd3fff640 1 -- 172.21.15.19:0/1445765033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be0009c30 con 0x7f8bec109b10 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.400+0000 7f8bd3fff640 1 -- 172.21.15.19:0/1445765033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be000f620 con 0x7f8bec109b10 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.400+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec19f830 con 0x7f8bec109b10 2024-04-25T11:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.400+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec19fd30 con 0x7f8bec109b10 2024-04-25T11:03:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.402+0000 7f8bf8484740 1 Processor -- start 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.402+0000 7f8bf8484740 1 -- start start 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.402+0000 7f8bd3fff640 1 -- 172.21.15.19:0/1445765033 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8be001a070 con 0x7f8bec109b10 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.402+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc3fe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc035550 0x55f4cc3feb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc03d730 0x55f4cc3ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bf8484740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc364b00 con 0x55f4cc035550 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bf8484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc300920 con 0x55f4cc03d730 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bf8484740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc360ac0 con 0x55f4cc05e410 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc035550 0x55f4cc3feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc035550 0x55f4cc3feb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48640/0 (socket says 172.21.15.19:48640) 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 -- 172.21.15.19:0/2825266920 learned_addr learned my addr 172.21.15.19:0/2825266920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc03d730 0x55f4cc3ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc3fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 -- 172.21.15.19:0/2825266920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 msgr2=0x55f4cc3fe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc3fe620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 -- 172.21.15.19:0/2825266920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc03d730 msgr2=0x55f4cc3ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc03d730 0x55f4cc3ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 -- 172.21.15.19:0/2825266920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4cc13e7e0 con 0x55f4cc035550 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc3fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bd3fff640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8bd40afc60 0x7f8bd40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beaffd640 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc035550 0x55f4cc3feb60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009c90 tx=0x7f8bdc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e9dbc2eb0efa95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8bd40afc60 0x7f8bd40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bd3fff640 1 -- 172.21.15.19:0/1445765033 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f8be00deae0 con 0x7f8bec109b10 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.403+0000 7f8bd17fa640 1 -- 172.21.15.19:0/2825266920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc01e070 con 0x55f4cc035550 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bd17fa640 1 -- 172.21.15.19:0/2825266920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc019ae0 con 0x55f4cc035550 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bd17fa640 1 -- 172.21.15.19:0/2825266920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00e940 con 0x55f4cc035550 2024-04-25T11:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 -- 172.21.15.19:0/2825266920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc035550 msgr2=0x55f4cc3feb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8bd40afc60 0x7f8bd40b2170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8be402f500 tx=0x7f8be402f310 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc035550 0x55f4cc3feb60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009c90 tx=0x7f8bdc011810 comp rx=0 tx=0).stop 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 -- 172.21.15.19:0/2825266920 shutdown_connections 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc03d730 0x55f4cc3ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc035550 0x55f4cc3feb60 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 --2- 172.21.15.19:0/2825266920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc3fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 -- 172.21.15.19:0/2825266920 >> 172.21.15.19:0/2825266920 conn(0x55f4cc052560 msgr2=0x55f4cc3fadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.404+0000 7f8bf8484740 1 -- 172.21.15.19:0/2825266920 shutdown_connections 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 -- 172.21.15.19:0/2825266920 wait complete. 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 -- start start 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 Processor -- start 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc408c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc409150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc414e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc40b970 con 0x55f4cc03d730 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc2fe5d0 con 0x55f4cc035550 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc40baf0 con 0x55f4cc05e410 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bf8484740 10 client.0 _ll_register_callbacks cb 0x55f4cc2ec080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc409150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc409150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48648/0 (socket says 172.21.15.19:48648) 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.405+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3842533639 learned_addr learned my addr 172.21.15.19:0/3842533639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8beaffd640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc408c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc414e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3842533639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 msgr2=0x55f4cc414e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc414e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 msgr2=0x55f4cc408c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc408c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3842533639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4cc19ef70 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8beaffd640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc408c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc409150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8be00a2ca0 tx=0x7f8be00e20c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4302a2d5bb8e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bceff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bceff5640 1 -- 172.21.15.19:0/3842533639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0021070 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bceff5640 1 -- 172.21.15.19:0/3842533639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f8be00a6090 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bdc019ae0 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bceff5640 1 -- 172.21.15.19:0/3842533639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8be00a6570 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bf8484740 10 client.16944 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f4cc036170 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.406+0000 7f8bceff5640 1 -- 172.21.15.19:0/3842533639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc019ae0 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bceff5640 1 -- 172.21.15.19:0/3842533639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8be00a58f0 con 0x55f4cc03d730 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bf8484740 10 client.16944 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bf8484740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bf8484740 20 client.16944 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cc01d1f0 0x55f4cc41af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bf8484740 4 client.16944 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4cc372710 con 0x55f4cc01d1f0 2024-04-25T11:03:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8beaffd640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cc01d1f0 0x55f4cc41af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8beaffd640 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cc01d1f0 0x55f4cc41af20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526634d91e97acd8 server_cookie=e1c76d50d099847f in_seq=0 out_seq=0 2024-04-25T11:03:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.407+0000 7f8bceff5640 10 client.16944 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bceff5640 1 -- 172.21.15.19:0/3842533639 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4cc372710 con 0x55f4cc01d1f0 2024-04-25T11:03:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bceff5640 10 client.16944 handle_command_reply: tid=0 2024-04-25T11:03:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 client.16944 shutdown 2024-04-25T11:03:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 msgr2=0x55f4cc409150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc409150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8be00a2ca0 tx=0x7f8be00e20c0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 shutdown_connections 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cc01d1f0 0x55f4cc41af20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc414e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc409150 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bceff5640 0 client.16944 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3842533639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc408c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.408+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 >> 172.21.15.19:0/3842533639 conn(0x55f4cc052560 msgr2=0x55f4cc400340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.409+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 shutdown_connections 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.409+0000 7f8bf8484740 1 -- 172.21.15.19:0/3842533639 wait complete. 2024-04-25T11:03:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.409+0000 7f8bf8484740 20 client.16944 trim_cache size 0 max 16384 2024-04-25T11:03:41.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 Processor -- start 2024-04-25T11:03:41.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 -- start start 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc3fae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc4206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc420c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ada80 con 0x55f4cc03d730 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1b02e0 con 0x55f4cc035550 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bf8484740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ae0d0 con 0x55f4cc05e410 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc4206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc4206e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48650/0 (socket says 172.21.15.19:48650) 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.427+0000 7f8bea7fc640 1 -- 172.21.15.19:0/16683086 learned_addr learned my addr 172.21.15.19:0/16683086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8beaffd640 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc3fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bea7fc640 1 -- 172.21.15.19:0/16683086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 msgr2=0x55f4cc420c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc420c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc420c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bea7fc640 1 -- 172.21.15.19:0/16683086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 msgr2=0x55f4cc3fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc3fae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bea7fc640 1 -- 172.21.15.19:0/16683086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4cc3ffc10 con 0x55f4cc03d730 2024-04-25T11:03:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8beaffd640 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc3fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc4206e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8be00a5cb0 tx=0x7f8be00a24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c146bc1410b3f086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bd0ff9640 1 -- 172.21.15.19:0/16683086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be00bf030 con 0x55f4cc03d730 2024-04-25T11:03:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bd0ff9640 1 -- 172.21.15.19:0/16683086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be00e5060 con 0x55f4cc03d730 2024-04-25T11:03:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.428+0000 7f8bd0ff9640 1 -- 172.21.15.19:0/16683086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0021070 con 0x55f4cc03d730 2024-04-25T11:03:41.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 -- 172.21.15.19:0/16683086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 msgr2=0x55f4cc4206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc4206e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8be00a5cb0 tx=0x7f8be00a24e0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 -- 172.21.15.19:0/16683086 shutdown_connections 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc05e410 0x55f4cc420c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc4206e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 --2- 172.21.15.19:0/16683086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc035550 0x55f4cc3fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 -- 172.21.15.19:0/16683086 >> 172.21.15.19:0/16683086 conn(0x55f4cc052560 msgr2=0x55f4cc3fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 -- 172.21.15.19:0/16683086 shutdown_connections 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 -- 172.21.15.19:0/16683086 wait complete. 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 -- start start 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.429+0000 7f8bf8484740 1 Processor -- start 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc035550 0x55f4cc40c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc40cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc05e410 0x55f4cc40d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ae7a0 con 0x55f4cc03d730 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1afc10 con 0x55f4cc05e410 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1aee70 con 0x55f4cc035550 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 10 client.0 _ll_register_callbacks cb 0x7f8be00a30e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc40cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc40cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48664/0 (socket says 172.21.15.19:48664) 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3680262035 learned_addr learned my addr 172.21.15.19:0/3680262035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8beb7fe640 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc05e410 0x55f4cc40d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3680262035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc035550 msgr2=0x55f4cc40c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc035550 0x55f4cc40c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc05e410 msgr2=0x55f4cc40d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc05e410 0x55f4cc40d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 -- 172.21.15.19:0/3680262035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4cc4866d0 con 0x55f4cc03d730 2024-04-25T11:03:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bea7fc640 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc40cf30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8be00a6920 tx=0x7f8be00a7320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5810588dd24f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bceff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bceff5640 1 -- 172.21.15.19:0/3680262035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be00bf070 con 0x55f4cc03d730 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bceff5640 1 -- 172.21.15.19:0/3680262035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f8be00a27a0 con 0x55f4cc03d730 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.430+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be00e5060 con 0x55f4cc03d730 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bceff5640 1 -- 172.21.15.19:0/3680262035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8be0021070 con 0x55f4cc03d730 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 10 client.16950 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f4cc4878e0 con 0x55f4cc03d730 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bceff5640 1 -- 172.21.15.19:0/3680262035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be00e5060 con 0x55f4cc03d730 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bceff5640 1 -- 172.21.15.19:0/3680262035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8be00c90e0 con 0x55f4cc03d730 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 10 client.16950 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 20 client.16950 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cbff2630 0x55f4cc489f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 4 client.16950 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f4cc36c580 con 0x55f4cbff2630 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.431+0000 7f8beaffd640 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cbff2630 0x55f4cc489f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.432+0000 7f8bceff5640 10 client.16950 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.432+0000 7f8beaffd640 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cbff2630 0x55f4cc489f50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaf5d6a737ea3dd3 server_cookie=b29503e360134ea2 in_seq=0 out_seq=0 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bceff5640 1 -- 172.21.15.19:0/3680262035 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f4cc36c580 con 0x55f4cbff2630 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bceff5640 10 client.16950 handle_command_reply: tid=0 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 client.16950 shutdown 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 msgr2=0x55f4cc40cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc40cf30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8be00a6920 tx=0x7f8be00a7320 comp rx=0 tx=0).stop 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 shutdown_connections 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55f4cbff2630 0x55f4cc489f50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4cc05e410 0x55f4cc40d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4cc03d730 0x55f4cc40cf30 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 --2- 172.21.15.19:0/3680262035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc035550 0x55f4cc40c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bceff5640 0 client.16950 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 >> 172.21.15.19:0/3680262035 conn(0x55f4cc052560 msgr2=0x55f4cc3f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 shutdown_connections 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 1 -- 172.21.15.19:0/3680262035 wait complete. 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.433+0000 7f8bf8484740 20 client.16950 trim_cache size 0 max 16384 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "207f1dbc-2943-422b-ae46-ef01aab489a3", 2024-04-25T11:03:41.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:03:41.440 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:03:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8bd40afc60 msgr2=0x7f8bd40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8bd40afc60 0x7f8bd40b2170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8be402f500 tx=0x7f8be402f310 comp rx=0 tx=0).stop 2024-04-25T11:03:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec109b10 msgr2=0x7f8bec19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec109b10 0x7f8bec19ac40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8be0005d00 tx=0x7f8be000dde0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 shutdown_connections 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8bd40afc60 0x7f8bd40b2170 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bec109fb0 0x7f8bec19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bec109b10 0x7f8bec19ac40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 --2- 172.21.15.19:0/1445765033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bec104260 0x7f8bec19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 >> 172.21.15.19:0/1445765033 conn(0x7f8bec0ff230 msgr2=0x7f8bec106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 shutdown_connections 2024-04-25T11:03:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.434+0000 7f8bf15ce640 1 -- 172.21.15.19:0/1445765033 wait complete. 2024-04-25T11:03:41.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:03:41.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:03:41.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:41 smithi169 ceph-mon[33049]: pgmap v284: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:41.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:41 smithi112 ceph-mon[32799]: pgmap v284: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.559+0000 7f3f9f59e640 1 -- 172.21.15.19:0/3135432483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0102b00 msgr2=0x7f3fa0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.559+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/3135432483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0102b00 0x7f3fa0102f10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009900 tx=0x7f3f9002a410 comp rx=0 tx=0).stop 2024-04-25T11:03:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.559+0000 7f3f9f59e640 1 -- 172.21.15.19:0/3135432483 shutdown_connections 2024-04-25T11:03:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.559+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/3135432483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0107b50 0x7f3fa010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.559+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/3135432483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0107090 0x7f3fa01074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.559+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/3135432483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0102b00 0x7f3fa0102f10 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.559+0000 7f3f9f59e640 1 -- 172.21.15.19:0/3135432483 >> 172.21.15.19:0/3135432483 conn(0x7f3fa0078000 msgr2=0x7f3fa0078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.560+0000 7f3f9f59e640 1 -- 172.21.15.19:0/3135432483 shutdown_connections 2024-04-25T11:03:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.560+0000 7f3f9f59e640 1 -- 172.21.15.19:0/3135432483 wait complete. 2024-04-25T11:03:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.560+0000 7f3f9f59e640 1 Processor -- start 2024-04-25T11:03:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.560+0000 7f3f9f59e640 1 -- start start 2024-04-25T11:03:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102b00 0x7f3fa019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0107090 0x7f3fa019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0107b50 0x7f3fa019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9f59e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa0113b80 con 0x7f3fa0107b50 2024-04-25T11:03:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9f59e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa0113a00 con 0x7f3fa0107090 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9f59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa0113d00 con 0x7f3fa0102b00 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0107b50 0x7f3fa019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0107b50 0x7f3fa019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48690/0 (socket says 172.21.15.19:48690) 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 -- 172.21.15.19:0/1010872483 learned_addr learned my addr 172.21.15.19:0/1010872483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0107090 0x7f3fa019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102b00 0x7f3fa019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 -- 172.21.15.19:0/1010872483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102b00 msgr2=0x7f3fa019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102b00 0x7f3fa019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 -- 172.21.15.19:0/1010872483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0107090 msgr2=0x7f3fa019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0107090 0x7f3fa019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 -- 172.21.15.19:0/1010872483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa019f3e0 con 0x7f3fa0107b50 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.561+0000 7f3f9ed9d640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0107b50 0x7f3fa019eda0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800eaa0 tx=0x7f3f9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.562+0000 7f3f8f7fe640 1 -- 172.21.15.19:0/1010872483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9800cdb0 con 0x7f3fa0107b50 2024-04-25T11:03:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.562+0000 7f3f8f7fe640 1 -- 172.21.15.19:0/1010872483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98005020 con 0x7f3fa0107b50 2024-04-25T11:03:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.562+0000 7f3f8f7fe640 1 -- 172.21.15.19:0/1010872483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f980116d0 con 0x7f3fa0107b50 2024-04-25T11:03:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.562+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa019f6d0 con 0x7f3fa0107b50 2024-04-25T11:03:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.562+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa019fbd0 con 0x7f3fa0107b50 2024-04-25T11:03:41.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.564+0000 7f3fab98e740 1 Processor -- start 2024-04-25T11:03:41.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.564+0000 7f3f8f7fe640 1 -- 172.21.15.19:0/1010872483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3f980051d0 con 0x7f3fa0107b50 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.564+0000 7f3fab98e740 1 -- start start 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 0x55a535240060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a5352405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3fab98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5351a1d80 con 0x55a534e7a6b0 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3fab98e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a53515b8b0 con 0x55a534e724d0 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3fab98e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a53519dd40 con 0x55a534e9b390 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48706/0 (socket says 172.21.15.19:48706) 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1112228227 learned_addr learned my addr 172.21.15.19:0/1112228227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 0x55a535240060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a5352405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f8f7fe640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3f880afc50 0x7f3f880b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9ed9d640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3f880afc50 0x7f3f880b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1112228227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 msgr2=0x55a535240060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 0x55a535240060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1112228227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 msgr2=0x55a5352405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a5352405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1112228227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53512ca40 con 0x55a534e7a6b0 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f8f7fe640 1 -- 172.21.15.19:0/1010872483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f3f980df1b0 con 0x7f3fa0107b50 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.565+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524cef0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f94005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08ac9be0d5d4d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.566+0000 7f3f8cff9640 1 -- 172.21.15.19:0/1112228227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94004030 con 0x55a534e7a6b0 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.566+0000 7f3f9ed9d640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3f880afc50 0x7f3f880b2160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800cfd0 tx=0x7f3f98099040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.566+0000 7f3f8cff9640 1 -- 172.21.15.19:0/1112228227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9400fc40 con 0x55a534e7a6b0 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.566+0000 7f3f8cff9640 1 -- 172.21.15.19:0/1112228227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94003830 con 0x55a534e7a6b0 2024-04-25T11:03:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 -- 172.21.15.19:0/1112228227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 msgr2=0x55a53524cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524cef0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f94005850 comp rx=0 tx=0).stop 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 -- 172.21.15.19:0/1112228227 shutdown_connections 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524cef0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a5352405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1112228227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 0x55a535240060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 -- 172.21.15.19:0/1112228227 >> 172.21.15.19:0/1112228227 conn(0x55a534e8f4e0 msgr2=0x55a53523c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 -- 172.21.15.19:0/1112228227 shutdown_connections 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 -- 172.21.15.19:0/1112228227 wait complete. 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 -- start start 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.567+0000 7f3fab98e740 1 Processor -- start 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e724d0 0x55a53524ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e9b390 0x55a535256b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a53524d5e0 con 0x55a534e7a6b0 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a535128d10 con 0x55a534e9b390 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a53524d760 con 0x55a534e724d0 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 10 client.0 _ll_register_callbacks cb 0x55a535155210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48714/0 (socket says 172.21.15.19:48714) 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 -- 172.21.15.19:0/2230631960 learned_addr learned my addr 172.21.15.19:0/2230631960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e724d0 0x55a53524ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e9b390 0x55a535256b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 -- 172.21.15.19:0/2230631960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e724d0 msgr2=0x55a53524ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e724d0 0x55a53524ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 -- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e9b390 msgr2=0x55a535256b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e9b390 0x55a535256b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 -- 172.21.15.19:0/2230631960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a534f7d7d0 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e9b390 0x55a535256b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524b040 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3f9002a930 tx=0x7f3f900314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481e6d5c6ac342d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f85ffb640 1 -- 172.21.15.19:0/2230631960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90035040 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9400fc40 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.568+0000 7f3f85ffb640 1 -- 172.21.15.19:0/2230631960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f3f90031810 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3f85ffb640 1 -- 172.21.15.19:0/2230631960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3f90004040 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 10 client.16962 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a534eb33f0 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3f85ffb640 1 -- 172.21.15.19:0/2230631960 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9400fc40 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3f85ffb640 1 -- 172.21.15.19:0/2230631960 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3f90004910 con 0x55a534e7a6b0 2024-04-25T11:03:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 10 client.16962 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 20 client.16962 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e59f60 0x55a53525ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 4 client.16962 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5351af990 con 0x55a534e59f60 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.569+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e59f60 0x55a53525ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.570+0000 7f3f85ffb640 10 client.16962 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.570+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e59f60 0x55a53525ccc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2446c995b59e6e39 server_cookie=51ea7f322ce0b513 in_seq=0 out_seq=0 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3f85ffb640 1 -- 172.21.15.19:0/2230631960 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5351af990 con 0x55a534e59f60 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3f85ffb640 10 client.16962 handle_command_reply: tid=0 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 client.16962 shutdown 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 msgr2=0x55a53524b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524b040 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3f9002a930 tx=0x7f3f900314a0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 shutdown_connections 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e59f60 0x55a53525ccc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e9b390 0x55a535256b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e7a6b0 0x55a53524b040 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3f85ffb640 0 client.16962 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 --2- 172.21.15.19:0/2230631960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e724d0 0x55a53524ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 >> 172.21.15.19:0/2230631960 conn(0x55a534e8f4e0 msgr2=0x55a535241d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 shutdown_connections 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 1 -- 172.21.15.19:0/2230631960 wait complete. 2024-04-25T11:03:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.571+0000 7f3fab98e740 20 client.16962 trim_cache size 0 max 16384 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 Processor -- start 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 -- start start 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a535244f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e7a6b0 0x55a5352624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e9b390 0x55a535262a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01ab600 con 0x55a534e9b390 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01ade40 con 0x55a534e724d0 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3fab98e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01abc50 con 0x55a534e7a6b0 2024-04-25T11:03:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e9b390 0x55a535262a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e9b390 0x55a535262a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48716/0 (socket says 172.21.15.19:48716) 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/3221780905 learned_addr learned my addr 172.21.15.19:0/3221780905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.589+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a535244f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e7a6b0 0x55a5352624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/3221780905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e7a6b0 msgr2=0x55a5352624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e7a6b0 0x55a5352624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/3221780905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 msgr2=0x55a535244f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a535244f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/3221780905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5352461b0 con 0x55a534e9b390 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e9b390 0x55a535262a00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f94005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67013f8d6a19d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f87fff640 1 -- 172.21.15.19:0/3221780905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9402f020 con 0x55a534e9b390 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f87fff640 1 -- 172.21.15.19:0/3221780905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9400f790 con 0x55a534e9b390 2024-04-25T11:03:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.590+0000 7f3f87fff640 1 -- 172.21.15.19:0/3221780905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94029690 con 0x55a534e9b390 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- 172.21.15.19:0/3221780905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e9b390 msgr2=0x55a535262a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e9b390 0x55a535262a00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f94005f90 comp rx=0 tx=0).stop 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- 172.21.15.19:0/3221780905 shutdown_connections 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e9b390 0x55a535262a00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e7a6b0 0x55a5352624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 --2- 172.21.15.19:0/3221780905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e724d0 0x55a535244f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- 172.21.15.19:0/3221780905 >> 172.21.15.19:0/3221780905 conn(0x55a534e8f4e0 msgr2=0x55a53523fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- 172.21.15.19:0/3221780905 shutdown_connections 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- 172.21.15.19:0/3221780905 wait complete. 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- start start 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 Processor -- start 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e724d0 0x55a535252dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e7a6b0 0x55a53524e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 0x55a53524e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01ac300 con 0x55a534e724d0 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01ad770 con 0x55a534e7a6b0 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01ac9d0 con 0x55a534e9b390 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.591+0000 7f3fab98e740 10 client.0 _ll_register_callbacks cb 0x7f3f90009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e724d0 0x55a535252dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e724d0 0x55a535252dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48724/0 (socket says 172.21.15.19:48724) 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1674720742 learned_addr learned my addr 172.21.15.19:0/1674720742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e7a6b0 0x55a53524e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1674720742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 msgr2=0x55a53524e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 0x55a53524e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e7a6b0 msgr2=0x55a53524e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e7a6b0 0x55a53524e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 -- 172.21.15.19:0/1674720742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a534e730f0 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f9dd9b640 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e724d0 0x55a535252dd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009f70 tx=0x7f3f9400faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692cf4c868430f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f85ffb640 1 -- 172.21.15.19:0/1674720742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94039060 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f85ffb640 1 -- 172.21.15.19:0/1674720742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f3f9400fc90 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9400f790 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3f85ffb640 1 -- 172.21.15.19:0/1674720742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3f9402f070 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3fab98e740 10 client.16968 fetch_fsmap learned FSMap version 20 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.592+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a5351624b0 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3f85ffb640 1 -- 172.21.15.19:0/1674720742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9400f790 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3f85ffb640 1 -- 172.21.15.19:0/1674720742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3f94004540 con 0x55a534e724d0 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3fab98e740 10 client.16968 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3fab98e740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:03:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3fab98e740 20 client.16968 populate_metadata read hostname 'smithi019' 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e2f3a0 0x55a5352c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3fab98e740 4 client.16968 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5351a9800 con 0x55a534e2f3a0 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e2f3a0 0x55a5352c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3f85ffb640 10 client.16968 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.593+0000 7f3f9e59c640 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e2f3a0 0x55a5352c6f00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85201fe1ebd4ddc server_cookie=a94e2e55f25221b7 in_seq=0 out_seq=0 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3f85ffb640 1 -- 172.21.15.19:0/1674720742 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a5351a9800 con 0x55a534e2f3a0 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3f85ffb640 10 client.16968 handle_command_reply: tid=0 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 client.16968 shutdown 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e724d0 msgr2=0x55a535252dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e724d0 0x55a535252dd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009f70 tx=0x7f3f9400faa0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 shutdown_connections 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a534e2f3a0 0x55a5352c6f00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a534e9b390 0x55a53524e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a534e7a6b0 0x55a53524e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3f85ffb640 0 client.16968 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 --2- 172.21.15.19:0/1674720742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a534e724d0 0x55a535252dd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 >> 172.21.15.19:0/1674720742 conn(0x55a534e8f4e0 msgr2=0x55a53523fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 shutdown_connections 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 1 -- 172.21.15.19:0/1674720742 wait complete. 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.594+0000 7f3fab98e740 20 client.16968 trim_cache size 0 max 16384 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:03:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:03:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "207f1dbc-2943-422b-ae46-ef01aab489a3": { 2024-04-25T11:03:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:03:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "207f1dbc-2943-422b-ae46-ef01aab489a3", 2024-04-25T11:03:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:03:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:03:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:03:41.602 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:03:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3f880afc50 msgr2=0x7f3f880b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3f880afc50 0x7f3f880b2160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800cfd0 tx=0x7f3f98099040 comp rx=0 tx=0).stop 2024-04-25T11:03:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0107b50 msgr2=0x7f3fa019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0107b50 0x7f3fa019eda0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800eaa0 tx=0x7f3f9800ef60 comp rx=0 tx=0).stop 2024-04-25T11:03:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 shutdown_connections 2024-04-25T11:03:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3f880afc50 0x7f3f880b2160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fa0107b50 0x7f3fa019eda0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0107090 0x7f3fa019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 --2- 172.21.15.19:0/1010872483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102b00 0x7f3fa019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:41.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 >> 172.21.15.19:0/1010872483 conn(0x7f3fa0078000 msgr2=0x7f3fa0100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:41.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 shutdown_connections 2024-04-25T11:03:41.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:03:41.595+0000 7f3f9f59e640 1 -- 172.21.15.19:0/1010872483 wait complete. 2024-04-25T11:03:41.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:41 smithi019 ceph-mon[29209]: pgmap v284: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:41.606 INFO:tasks.cephfs.filesystem:scrub status for tag:207f1dbc-2943-422b-ae46-ef01aab489a3 - {'path': '/', 'tag': '207f1dbc-2943-422b-ae46-ef01aab489a3', 'options': 'recursive,force'} 2024-04-25T11:03:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:42 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:03:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:42 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:42 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:03:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:42 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:42.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:42 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:03:42.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:42 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:03:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:42 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:42 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:03:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:42 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:42.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:42 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:03:42.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:42 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:03:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:42 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:42 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:03:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:42 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:03:42.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:42 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:03:43.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:43 smithi169 ceph-mon[33049]: pgmap v285: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:43.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:43 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:03:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:43 smithi112 ceph-mon[32799]: pgmap v285: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:43.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:43 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:03:43.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:43 smithi019 ceph-mon[29209]: pgmap v285: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:03:43.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:43 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:03:44.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:44 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:03:44.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:44 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:03:44.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:44 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:03:44.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:44 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:03:44.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:44 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:03:44.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:44 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:03:44.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:44 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:03:44.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:44 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:03:44.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:44 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:03:45.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:45 smithi169 ceph-mon[33049]: pgmap v286: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:45.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:45 smithi112 ceph-mon[32799]: pgmap v286: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:45.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:45 smithi019 ceph-mon[29209]: pgmap v286: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:47.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:47 smithi169 ceph-mon[33049]: pgmap v287: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:47.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:47 smithi112 ceph-mon[32799]: pgmap v287: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:47.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:47 smithi019 ceph-mon[29209]: pgmap v287: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:49.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:49 smithi169 ceph-mon[33049]: pgmap v288: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:49.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:49 smithi112 ceph-mon[32799]: pgmap v288: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:49.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:49 smithi019 ceph-mon[29209]: pgmap v288: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:51.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:51 smithi169 ceph-mon[33049]: pgmap v289: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:51.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:51 smithi112 ceph-mon[32799]: pgmap v289: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:51.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:51 smithi019 ceph-mon[29209]: pgmap v289: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:53.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:53 smithi169 ceph-mon[33049]: pgmap v290: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:53.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:53 smithi112 ceph-mon[32799]: pgmap v290: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:53.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:53 smithi019 ceph-mon[29209]: pgmap v290: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:55.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:55 smithi169 ceph-mon[33049]: pgmap v291: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:55.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:55 smithi112 ceph-mon[32799]: pgmap v291: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:55.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:55 smithi019 ceph-mon[29209]: pgmap v291: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-04-25T11:03:57.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:57 smithi112 ceph-mon[32799]: pgmap v292: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:57.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:57 smithi019 ceph-mon[29209]: pgmap v292: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:57.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:57 smithi169 ceph-mon[33049]: pgmap v292: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-25T11:03:59.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:03:59 smithi112 ceph-mon[32799]: pgmap v293: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:59.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:03:59 smithi019 ceph-mon[29209]: pgmap v293: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:03:59.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:03:59 smithi169 ceph-mon[33049]: pgmap v293: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:01.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:01 smithi112 ceph-mon[32799]: pgmap v294: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:01.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:01 smithi019 ceph-mon[29209]: pgmap v294: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:01.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:01 smithi169 ceph-mon[33049]: pgmap v294: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:03.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:03 smithi112 ceph-mon[32799]: pgmap v295: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T11:04:03.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:03 smithi019 ceph-mon[29209]: pgmap v295: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T11:04:03.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:03 smithi169 ceph-mon[33049]: pgmap v295: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T11:04:05.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:05 smithi112 ceph-mon[32799]: pgmap v296: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T11:04:05.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:05 smithi019 ceph-mon[29209]: pgmap v296: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T11:04:05.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:05 smithi169 ceph-mon[33049]: pgmap v296: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T11:04:07.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:07 smithi112 ceph-mon[32799]: pgmap v297: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:04:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:07 smithi019 ceph-mon[29209]: pgmap v297: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:04:07.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:07 smithi169 ceph-mon[33049]: pgmap v297: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T11:04:08.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:08 smithi112 ceph-mon[32799]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:04:08.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:08 smithi019 ceph-mon[29209]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:04:08.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:08 smithi169 ceph-mon[33049]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:04:09.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:09 smithi112 ceph-mon[32799]: pgmap v299: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:04:09.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:09 smithi019 ceph-mon[29209]: pgmap v299: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:04:09.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:09 smithi169 ceph-mon[33049]: pgmap v299: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:04:11.553 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:11 smithi169 ceph-mon[33049]: pgmap v300: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:04:11.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:11 smithi112 ceph-mon[32799]: pgmap v300: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:04:11.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:04:11.607 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:04:11.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:11 smithi019 ceph-mon[29209]: pgmap v300: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-04-25T11:04:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 -- 172.21.15.19:0/1614899986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 msgr2=0x7f85981025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 --2- 172.21.15.19:0/1614899986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f85981025b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8588009ca0 tx=0x7f858802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 -- 172.21.15.19:0/1614899986 shutdown_connections 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 --2- 172.21.15.19:0/1614899986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f859810d380 0x7f859810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 --2- 172.21.15.19:0/1614899986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8598102af0 0x7f8598102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 --2- 172.21.15.19:0/1614899986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f85981025b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 -- 172.21.15.19:0/1614899986 >> 172.21.15.19:0/1614899986 conn(0x7f8598078100 msgr2=0x7f8598078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 -- 172.21.15.19:0/1614899986 shutdown_connections 2024-04-25T11:04:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.736+0000 7f859d030640 1 -- 172.21.15.19:0/1614899986 wait complete. 2024-04-25T11:04:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 Processor -- start 2024-04-25T11:04:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 -- start start 2024-04-25T11:04:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8598102af0 0x7f859819b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f859819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f859810d380 0x7f859819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8598112be0 con 0x7f8598108a20 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8598112a60 con 0x7f8598102af0 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859d030640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8598112d60 con 0x7f859810d380 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f859819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f859819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35212/0 (socket says 172.21.15.19:35212) 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.737+0000 7f859659c640 1 -- 172.21.15.19:0/3127290327 learned_addr learned my addr 172.21.15.19:0/3127290327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f8596d9d640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8598102af0 0x7f859819b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859759e640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f859810d380 0x7f859819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859659c640 1 -- 172.21.15.19:0/3127290327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f859810d380 msgr2=0x7f859819c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859659c640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f859810d380 0x7f859819c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859659c640 1 -- 172.21.15.19:0/3127290327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8598102af0 msgr2=0x7f859819b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859659c640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8598102af0 0x7f859819b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859659c640 1 -- 172.21.15.19:0/3127290327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981a6890 con 0x7f8598108a20 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859659c640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f859819bac0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f858c005fd0 tx=0x7f858c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f857ffff640 1 -- 172.21.15.19:0/3127290327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c017070 con 0x7f8598108a20 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981a6b80 con 0x7f8598108a20 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f857ffff640 1 -- 172.21.15.19:0/3127290327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c009c30 con 0x7f8598108a20 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f857ffff640 1 -- 172.21.15.19:0/3127290327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c003d10 con 0x7f8598108a20 2024-04-25T11:04:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.738+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981a70d0 con 0x7f8598108a20 2024-04-25T11:04:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.741+0000 7f85a3fb7740 1 Processor -- start 2024-04-25T11:04:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.741+0000 7f85a3fb7740 1 -- start start 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0f34d0 0x561fab4bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561fab422d80 con 0x561fab0fb6b0 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561fab3dc8b0 con 0x561fab11c390 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561fab41ed40 con 0x561fab0f34d0 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f857ffff640 1 -- 172.21.15.19:0/3127290327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f858c010460 con 0x7f8598108a20 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f859759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35226/0 (socket says 172.21.15.19:35226) 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f859659c640 1 -- 172.21.15.19:0/1364145563 learned_addr learned my addr 172.21.15.19:0/1364145563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.742+0000 7f8596d9d640 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0f34d0 0x561fab4bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859659c640 1 -- 172.21.15.19:0/1364145563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0f34d0 msgr2=0x561fab4bcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859659c640 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0f34d0 0x561fab4bcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859659c640 1 -- 172.21.15.19:0/1364145563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 msgr2=0x561fab4bca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859659c640 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4bca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859659c640 1 -- 172.21.15.19:0/1364145563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fab3ada40 con 0x561fab0fb6b0 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859759e640 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f8596d9d640 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0f34d0 0x561fab4bcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f857ffff640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f85800afc60 0x7f85800b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859659c640 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4c98f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f858c0dc2e0 tx=0x7f858c0e7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df74abe5cbb3d82f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f859759e640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f85800afc60 0x7f85800b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f857ffff640 1 -- 172.21.15.19:0/3127290327 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8557+0+0 (secure 0 0 0) 0x7f858c0dec40 con 0x7f8598108a20 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f857d7fa640 1 -- 172.21.15.19:0/1364145563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c0a2c80 con 0x561fab0fb6b0 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f857d7fa640 1 -- 172.21.15.19:0/1364145563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c0ec070 con 0x561fab0fb6b0 2024-04-25T11:04:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.743+0000 7f857d7fa640 1 -- 172.21.15.19:0/1364145563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c0a6530 con 0x561fab0fb6b0 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 -- 172.21.15.19:0/1364145563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 msgr2=0x561fab4c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4c98f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f858c0dc2e0 tx=0x7f858c0e7780 comp rx=0 tx=0).stop 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f859759e640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f85800afc60 0x7f85800b2170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x561fab4b9bd0 tx=0x7f8590005cc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 -- 172.21.15.19:0/1364145563 shutdown_connections 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4c98f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0f34d0 0x561fab4bcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/1364145563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 -- 172.21.15.19:0/1364145563 >> 172.21.15.19:0/1364145563 conn(0x561fab1104e0 msgr2=0x561fab4b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 -- 172.21.15.19:0/1364145563 shutdown_connections 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 -- 172.21.15.19:0/1364145563 wait complete. 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.744+0000 7f85a3fb7740 1 -- start start 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 1 Processor -- start 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0f34d0 0x561fab4c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0fb6b0 0x561fab4c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561fab4c9fe0 con 0x561fab0f34d0 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561fab3a9d10 con 0x561fab11c390 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561fab4ca160 con 0x561fab0fb6b0 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f85a3fb7740 10 client.0 _ll_register_callbacks cb 0x561fab3d6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0f34d0 0x561fab4c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0f34d0 0x561fab4c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35238/0 (socket says 172.21.15.19:35238) 2024-04-25T11:04:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 -- 172.21.15.19:0/2352539939 learned_addr learned my addr 172.21.15.19:0/2352539939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f859759e640 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f859659c640 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0fb6b0 0x561fab4c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 -- 172.21.15.19:0/2352539939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0fb6b0 msgr2=0x561fab4c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0fb6b0 0x561fab4c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 -- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 msgr2=0x561fab4d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4d34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 -- 172.21.15.19:0/2352539939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fab1fe7d0 con 0x561fab0f34d0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.745+0000 7f8596d9d640 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0f34d0 0x561fab4c7580 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8598103a20 tx=0x7f8588031470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a524c87791f49c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f857aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f857aff5640 1 -- 172.21.15.19:0/2352539939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588031830 con 0x561fab0f34d0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f857aff5640 1 -- 172.21.15.19:0/2352539939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8557+0+0 (secure 0 0 0) 0x7f8588003970 con 0x561fab0f34d0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f858c0ec070 con 0x561fab0f34d0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f857aff5640 1 -- 172.21.15.19:0/2352539939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8588003b70 con 0x561fab0f34d0 2024-04-25T11:04:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 10 client.16980 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561fab1343f0 con 0x561fab0f34d0 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f857aff5640 1 -- 172.21.15.19:0/2352539939 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c0ec070 con 0x561fab0f34d0 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f857aff5640 1 -- 172.21.15.19:0/2352539939 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f858802f660 con 0x561fab0f34d0 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 10 client.16980 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 20 client.16980 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0daf60 0x561fab4d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 4 client.16980 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.746+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fab430990 con 0x561fab0daf60 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.747+0000 7f859659c640 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0daf60 0x561fab4d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.747+0000 7f859659c640 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0daf60 0x561fab4d9640 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3cc5ae38bb3699 server_cookie=6cad57719ddc1307 in_seq=0 out_seq=0 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.747+0000 7f857aff5640 10 client.16980 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f857aff5640 1 -- 172.21.15.19:0/2352539939 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fab430990 con 0x561fab0daf60 2024-04-25T11:04:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f857aff5640 10 client.16980 handle_command_reply: tid=0 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 client.16980 shutdown 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0f34d0 msgr2=0x561fab4c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0f34d0 0x561fab4c7580 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8598103a20 tx=0x7f8588031470 comp rx=0 tx=0).stop 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 shutdown_connections 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0daf60 0x561fab4d9640 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab11c390 0x561fab4d34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f857aff5640 0 client.16980 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab0fb6b0 0x561fab4c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/2352539939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0f34d0 0x561fab4c7580 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 >> 172.21.15.19:0/2352539939 conn(0x561fab1104e0 msgr2=0x561fab4be780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 shutdown_connections 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 1 -- 172.21.15.19:0/2352539939 wait complete. 2024-04-25T11:04:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.748+0000 7f85a3fb7740 20 client.16980 trim_cache size 0 max 16384 2024-04-25T11:04:11.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 Processor -- start 2024-04-25T11:04:11.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 -- start start 2024-04-25T11:04:11.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 0x561fab4df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85981ad900 con 0x561fab0fb6b0 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85981b0110 con 0x561fab0f34d0 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85981adf50 con 0x561fab11c390 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4dee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35248/0 (socket says 172.21.15.19:35248) 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 -- 172.21.15.19:0/682820026 learned_addr learned my addr 172.21.15.19:0/682820026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f8596d9d640 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 -- 172.21.15.19:0/682820026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 msgr2=0x561fab4df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 0x561fab4df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 -- 172.21.15.19:0/682820026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 msgr2=0x561fab4c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f859659c640 1 -- 172.21.15.19:0/682820026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fab0f40f0 con 0x561fab0fb6b0 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.767+0000 7f8596d9d640 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f859659c640 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4dee00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f858c0dc970 tx=0x7f858c0e2790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f510ff6e8c1900 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f857cff9640 1 -- 172.21.15.19:0/682820026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c08d070 con 0x561fab0fb6b0 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f857cff9640 1 -- 172.21.15.19:0/682820026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c0e2560 con 0x561fab0fb6b0 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f857cff9640 1 -- 172.21.15.19:0/682820026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c0a6710 con 0x561fab0fb6b0 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 -- 172.21.15.19:0/682820026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 msgr2=0x561fab4dee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4dee00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f858c0dc970 tx=0x7f858c0e2790 comp rx=0 tx=0).stop 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 -- 172.21.15.19:0/682820026 shutdown_connections 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 0x561fab4df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4dee00 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/682820026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 -- 172.21.15.19:0/682820026 >> 172.21.15.19:0/682820026 conn(0x561fab1104e0 msgr2=0x561fab4becd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 -- 172.21.15.19:0/682820026 shutdown_connections 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.768+0000 7f85a3fb7740 1 -- 172.21.15.19:0/682820026 wait complete. 2024-04-25T11:04:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 -- start start 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 Processor -- start 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 0x561fab4cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85981ae5d0 con 0x561fab0fb6b0 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85981afa40 con 0x561fab0f34d0 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85981aeca0 con 0x561fab11c390 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f85a3fb7740 10 client.0 _ll_register_callbacks cb 0x7f858802f3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f859659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35254/0 (socket says 172.21.15.19:35254) 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f859659c640 1 -- 172.21.15.19:0/409949787 learned_addr learned my addr 172.21.15.19:0/409949787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f8596d9d640 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.769+0000 7f859759e640 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 0x561fab4cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f859659c640 1 -- 172.21.15.19:0/409949787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 msgr2=0x561fab4cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f859659c640 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 0x561fab4cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f859659c640 1 -- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 msgr2=0x561fab4cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f859659c640 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f859659c640 1 -- 172.21.15.19:0/409949787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fab3e34b0 con 0x561fab0fb6b0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f859659c640 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4cab50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f858c0a4030 tx=0x7f858c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4212a824a6b268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f857aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f857aff5640 1 -- 172.21.15.19:0/409949787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c08d070 con 0x561fab0fb6b0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f857aff5640 1 -- 172.21.15.19:0/409949787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8557+0+0 (secure 0 0 0) 0x7f858c0a33b0 con 0x561fab0fb6b0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f858c0e2560 con 0x561fab0fb6b0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f857aff5640 1 -- 172.21.15.19:0/409949787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f858c0a6b90 con 0x561fab0fb6b0 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f85a3fb7740 10 client.16986 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561fab541490 con 0x561fab0fb6b0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f857aff5640 1 -- 172.21.15.19:0/409949787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c0e2560 con 0x561fab0fb6b0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f857aff5640 1 -- 172.21.15.19:0/409949787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f858c0dc5f0 con 0x561fab0fb6b0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f85a3fb7740 10 client.16986 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.770+0000 7f85a3fb7740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.771+0000 7f85a3fb7740 20 client.16986 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.771+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0b03a0 0x561fab5436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.771+0000 7f85a3fb7740 4 client.16986 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.771+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fab42a800 con 0x561fab0b03a0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.771+0000 7f8596d9d640 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0b03a0 0x561fab5436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.771+0000 7f857aff5640 10 client.16986 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.771+0000 7f8596d9d640 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0b03a0 0x561fab5436c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9daff1449d80542 server_cookie=20a4a186ae3f2929 in_seq=0 out_seq=0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f857aff5640 1 -- 172.21.15.19:0/409949787 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fab42a800 con 0x561fab0b03a0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f857aff5640 10 client.16986 handle_command_reply: tid=0 2024-04-25T11:04:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 client.16986 shutdown 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 msgr2=0x561fab4cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4cab50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f858c0a4030 tx=0x7f858c005d30 comp rx=0 tx=0).stop 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 shutdown_connections 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x561fab0b03a0 0x561fab5436c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fab11c390 0x561fab4cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fab0fb6b0 0x561fab4cab50 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 --2- 172.21.15.19:0/409949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fab0f34d0 0x561fab4cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f857aff5640 0 client.16986 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 >> 172.21.15.19:0/409949787 conn(0x561fab1104e0 msgr2=0x561fab4becd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 shutdown_connections 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 1 -- 172.21.15.19:0/409949787 wait complete. 2024-04-25T11:04:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f85a3fb7740 20 client.16986 trim_cache size 0 max 16384 2024-04-25T11:04:11.779 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:11.779 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:04:11.779 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:04:11.779 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f85800afc60 msgr2=0x7f85800b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f85800afc60 0x7f85800b2170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x561fab4b9bd0 tx=0x7f8590005cc0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 msgr2=0x7f859819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f859819bac0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f858c005fd0 tx=0x7f858c00dde0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 shutdown_connections 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f85800afc60 0x7f85800b2170 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f859810d380 0x7f859819c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8598108a20 0x7f859819bac0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 --2- 172.21.15.19:0/3127290327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8598102af0 0x7f859819b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.772+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 >> 172.21.15.19:0/3127290327 conn(0x7f8598078100 msgr2=0x7f85981040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.773+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 shutdown_connections 2024-04-25T11:04:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.773+0000 7f859d030640 1 -- 172.21.15.19:0/3127290327 wait complete. 2024-04-25T11:04:11.784 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:04:11.784 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:04:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 -- 172.21.15.19:0/3945770941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84104280 msgr2=0x7f3d84110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/3945770941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84104280 0x7f3d84110650 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009ca0 tx=0x7f3d7c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 -- 172.21.15.19:0/3945770941 shutdown_connections 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/3945770941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84104280 0x7f3d84110650 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/3945770941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d84109fd0 0x7f3d84103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/3945770941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84109b30 0x7f3d841036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 -- 172.21.15.19:0/3945770941 >> 172.21.15.19:0/3945770941 conn(0x7f3d840ff210 msgr2=0x7f3d84101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 -- 172.21.15.19:0/3945770941 shutdown_connections 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.896+0000 7f3d8aee0640 1 -- 172.21.15.19:0/3945770941 wait complete. 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 Processor -- start 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 -- start start 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84104280 0x7f3d8419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84109b30 0x7f3d8419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d84109fd0 0x7f3d8419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84113d90 con 0x7f3d84109b30 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84113c10 con 0x7f3d84104280 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8aee0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84113f10 con 0x7f3d84109fd0 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84109b30 0x7f3d8419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84109b30 0x7f3d8419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35268/0 (socket says 172.21.15.19:35268) 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d83fff640 1 -- 172.21.15.19:0/246618273 learned_addr learned my addr 172.21.15.19:0/246618273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d8947d640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d84109fd0 0x7f3d8419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.897+0000 7f3d88c7c640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84104280 0x7f3d8419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d83fff640 1 -- 172.21.15.19:0/246618273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d84109fd0 msgr2=0x7f3d8419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d83fff640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d84109fd0 0x7f3d8419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d83fff640 1 -- 172.21.15.19:0/246618273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84104280 msgr2=0x7f3d8419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d83fff640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84104280 0x7f3d8419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d83fff640 1 -- 172.21.15.19:0/246618273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8419f610 con 0x7f3d84109b30 2024-04-25T11:04:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d8947d640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d84109fd0 0x7f3d8419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d88c7c640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84104280 0x7f3d8419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d83fff640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84109b30 0x7f3d8419ad10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005d00 tx=0x7f3d7000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d81ffb640 1 -- 172.21.15.19:0/246618273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70017070 con 0x7f3d84109b30 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d81ffb640 1 -- 172.21.15.19:0/246618273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d70009c30 con 0x7f3d84109b30 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8419f900 con 0x7f3d84109b30 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d81ffb640 1 -- 172.21.15.19:0/246618273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000f620 con 0x7f3d84109b30 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.898+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8419fe00 con 0x7f3d84109b30 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.899+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8406a190 con 0x7f3d84109b30 2024-04-25T11:04:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.902+0000 7f3d81ffb640 1 -- 172.21.15.19:0/246618273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3d7001a070 con 0x7f3d84109b30 2024-04-25T11:04:11.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.902+0000 7f3d81ffb640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d740afc60 0x7f3d740b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:11.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.902+0000 7f3d81ffb640 1 -- 172.21.15.19:0/246618273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8557+0+0 (secure 0 0 0) 0x7f3d700deb00 con 0x7f3d84109b30 2024-04-25T11:04:11.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.902+0000 7f3d88c7c640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d740afc60 0x7f3d740b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:11.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.903+0000 7f3d88c7c640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d740afc60 0x7f3d740b2170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d84104810 tx=0x7f3d78005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:11.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:11.906+0000 7f3d81ffb640 1 -- 172.21.15.19:0/246618273 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d8419fe00 con 0x7f3d84109b30 2024-04-25T11:04:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.135+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d8419bb00 con 0x7f3d84109b30 2024-04-25T11:04:12.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.136+0000 7f3d81ffb640 1 -- 172.21.15.19:0/246618273 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f3d70009de0 con 0x7f3d84109b30 2024-04-25T11:04:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:04:12.137 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:04:12.137 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d740afc60 msgr2=0x7f3d740b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d740afc60 0x7f3d740b2170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d84104810 tx=0x7f3d78005f90 comp rx=0 tx=0).stop 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84109b30 msgr2=0x7f3d8419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84109b30 0x7f3d8419ad10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005d00 tx=0x7f3d7000dde0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 shutdown_connections 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f3d740afc60 0x7f3d740b2170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d84109fd0 0x7f3d8419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d84109b30 0x7f3d8419ad10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 --2- 172.21.15.19:0/246618273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d84104280 0x7f3d8419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 >> 172.21.15.19:0/246618273 conn(0x7f3d840ff210 msgr2=0x7f3d841019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 shutdown_connections 2024-04-25T11:04:12.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.138+0000 7f3d8aee0640 1 -- 172.21.15.19:0/246618273 wait complete. 2024-04-25T11:04:12.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 -- 172.21.15.19:0/3742472234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 msgr2=0x7fc65c101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 --2- 172.21.15.19:0/3742472234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c101b10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009900 tx=0x7fc64c02a410 comp rx=0 tx=0).stop 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 -- 172.21.15.19:0/3742472234 shutdown_connections 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 --2- 172.21.15.19:0/3742472234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65c10e550 0x7fc65c110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 --2- 172.21.15.19:0/3742472234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65c102050 0x7fc65c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 --2- 172.21.15.19:0/3742472234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c101b10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 -- 172.21.15.19:0/3742472234 >> 172.21.15.19:0/3742472234 conn(0x7fc65c077880 msgr2=0x7fc65c077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 -- 172.21.15.19:0/3742472234 shutdown_connections 2024-04-25T11:04:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.264+0000 7fc662851640 1 -- 172.21.15.19:0/3742472234 wait complete. 2024-04-25T11:04:12.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 Processor -- start 2024-04-25T11:04:12.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 -- start start 2024-04-25T11:04:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65c102050 0x7fc65c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65c10e550 0x7fc65c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c113d70 con 0x7fc65c109ba0 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c113bf0 con 0x7fc65c102050 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc662851640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c113ef0 con 0x7fc65c10e550 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c19ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35288/0 (socket says 172.21.15.19:35288) 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.265+0000 7fc65b7fe640 1 -- 172.21.15.19:0/2553758902 learned_addr learned my addr 172.21.15.19:0/2553758902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc65bfff640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65c102050 0x7fc65c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc660dee640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65c10e550 0x7fc65c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc65b7fe640 1 -- 172.21.15.19:0/2553758902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65c10e550 msgr2=0x7fc65c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65c10e550 0x7fc65c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc65b7fe640 1 -- 172.21.15.19:0/2553758902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65c102050 msgr2=0x7fc65c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65c102050 0x7fc65c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc65b7fe640 1 -- 172.21.15.19:0/2553758902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c19f610 con 0x7fc65c109ba0 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c19ad10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc65000b450 tx=0x7fc65000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc6597fa640 1 -- 172.21.15.19:0/2553758902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650015020 con 0x7fc65c109ba0 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc6597fa640 1 -- 172.21.15.19:0/2553758902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc650011d40 con 0x7fc65c109ba0 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c19f900 con 0x7fc65c109ba0 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc6597fa640 1 -- 172.21.15.19:0/2553758902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65000f340 con 0x7fc65c109ba0 2024-04-25T11:04:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.266+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1a7280 con 0x7fc65c109ba0 2024-04-25T11:04:12.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 Processor -- start 2024-04-25T11:04:12.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6597fa640 1 -- 172.21.15.19:0/2553758902 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc650018020 con 0x7fc65c109ba0 2024-04-25T11:04:12.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 -- start start 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ec17390 0x56119efb8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebee4d0 0x56119efb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119efc56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56119ef1dd80 con 0x56119ec17390 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56119eed7a20 con 0x56119ebee4d0 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6697e5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56119ef19d40 con 0x56119ebf66b0 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ec17390 0x56119efb8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ec17390 0x56119efb8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35296/0 (socket says 172.21.15.19:35296) 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 -- 172.21.15.19:0/2263861642 learned_addr learned my addr 172.21.15.19:0/2263861642 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc65bfff640 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebee4d0 0x56119efb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119efc56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 -- 172.21.15.19:0/2263861642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 msgr2=0x56119efc56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119efc56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 -- 172.21.15.19:0/2263861642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebee4d0 msgr2=0x56119efb8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebee4d0 0x56119efb8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc6597fa640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6440afce0 0x7fc6440b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.269+0000 7fc660dee640 1 -- 172.21.15.19:0/2263861642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119eee2c30 con 0x56119ec17390 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.270+0000 7fc6597fa640 1 -- 172.21.15.19:0/2553758902 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8557+0+0 (secure 0 0 0) 0x7fc6500de9b0 con 0x7fc65c109ba0 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.270+0000 7fc660dee640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6440afce0 0x7fc6440b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.270+0000 7fc660dee640 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ec17390 0x56119efb8810 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc654009eb0 tx=0x7fc654010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c0b2e48d3eff9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.270+0000 7fc6427fc640 1 -- 172.21.15.19:0/2263861642 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654021070 con 0x56119ec17390 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.270+0000 7fc6427fc640 1 -- 172.21.15.19:0/2263861642 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65401be40 con 0x56119ec17390 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.270+0000 7fc6427fc640 1 -- 172.21.15.19:0/2263861642 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654016070 con 0x56119ec17390 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 -- 172.21.15.19:0/2263861642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ec17390 msgr2=0x56119efb8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ec17390 0x56119efb8810 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc654009eb0 tx=0x7fc654010740 comp rx=0 tx=0).stop 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc660dee640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6440afce0 0x7fc6440b21f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc65401e000 tx=0x7fc654021000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:12.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 -- 172.21.15.19:0/2263861642 shutdown_connections 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119efc56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebee4d0 0x56119efb8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2263861642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ec17390 0x56119efb8810 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 -- 172.21.15.19:0/2263861642 >> 172.21.15.19:0/2263861642 conn(0x56119ec0b4e0 msgr2=0x56119efb4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 -- 172.21.15.19:0/2263861642 shutdown_connections 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 -- 172.21.15.19:0/2263861642 wait complete. 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.271+0000 7fc6697e5740 1 -- start start 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 1 Processor -- start 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119efc3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebf66b0 0x56119efc3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ec17390 0x56119efcf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56119efc5d90 con 0x56119ebee4d0 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56119eea4dc0 con 0x56119ebf66b0 2024-04-25T11:04:12.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56119efc5f10 con 0x56119ec17390 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc6697e5740 10 client.0 _ll_register_callbacks cb 0x56119eec0600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119efc3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119efc3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35312/0 (socket says 172.21.15.19:35312) 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 -- 172.21.15.19:0/2777588710 learned_addr learned my addr 172.21.15.19:0/2777588710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc660dee640 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ec17390 0x56119efcf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebf66b0 0x56119efc3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 -- 172.21.15.19:0/2777588710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ec17390 msgr2=0x56119efcf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ec17390 0x56119efcf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 -- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebf66b0 msgr2=0x56119efc3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebf66b0 0x56119efc3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.272+0000 7fc65bfff640 1 -- 172.21.15.19:0/2777588710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119ec2f3f0 con 0x56119ebee4d0 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc65bfff640 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119efc3240 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc65c069bf0 tx=0x7fc64c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a694af77bcc7971a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:12.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc63b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc63b7fe640 1 -- 172.21.15.19:0/2777588710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c035040 con 0x56119ebee4d0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc63b7fe640 1 -- 172.21.15.19:0/2777588710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8557+0+0 (secure 0 0 0) 0x7fc64c031d10 con 0x56119ebee4d0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56119ef2b990 con 0x56119ebee4d0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc63b7fe640 1 -- 172.21.15.19:0/2777588710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc64c004040 con 0x56119ebee4d0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc6697e5740 10 client.17004 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56119ecf97d0 con 0x56119ebee4d0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc63b7fe640 1 -- 172.21.15.19:0/2777588710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56119ef2b990 con 0x56119ebee4d0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.273+0000 7fc63b7fe640 1 -- 172.21.15.19:0/2777588710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc64c004750 con 0x56119ebee4d0 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc6697e5740 10 client.17004 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc6697e5740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc6697e5740 20 client.17004 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebd5f60 0x56119efd56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc6697e5740 4 client.17004 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:04:12.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56119ef25800 con 0x56119ebd5f60 2024-04-25T11:04:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebd5f60 0x56119efd56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc63b7fe640 10 client.17004 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.274+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebd5f60 0x56119efd56b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febbb90831f6ad63 server_cookie=289ac673704b24d6 in_seq=0 out_seq=0 2024-04-25T11:04:12.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc63b7fe640 1 -- 172.21.15.19:0/2777588710 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56119ef25800 con 0x56119ebd5f60 2024-04-25T11:04:12.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc63b7fe640 10 client.17004 handle_command_reply: tid=0 2024-04-25T11:04:12.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 client.17004 shutdown 2024-04-25T11:04:12.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 msgr2=0x56119efc3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119efc3240 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc65c069bf0 tx=0x7fc64c0316d0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 shutdown_connections 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebd5f60 0x56119efd56b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ec17390 0x56119efcf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc63b7fe640 0 client.17004 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ebf66b0 0x56119efc3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 --2- 172.21.15.19:0/2777588710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119efc3240 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.275+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 >> 172.21.15.19:0/2777588710 conn(0x56119ec0b4e0 msgr2=0x56119efba530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.276+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 shutdown_connections 2024-04-25T11:04:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.276+0000 7fc6697e5740 1 -- 172.21.15.19:0/2777588710 wait complete. 2024-04-25T11:04:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.276+0000 7fc6697e5740 20 client.17004 trim_cache size 0 max 16384 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.289+0000 7fc6697e5740 1 Processor -- start 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.289+0000 7fc6697e5740 1 -- start start 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebee4d0 0x56119efcee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebf66b0 0x56119efbab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efdae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc6697e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1adb20 con 0x56119ebf66b0 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc6697e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1b0380 con 0x56119ec17390 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc6697e5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1ae170 con 0x56119ebee4d0 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebf66b0 0x56119efbab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebf66b0 0x56119efbab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35320/0 (socket says 172.21.15.19:35320) 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 -- 172.21.15.19:0/844376485 learned_addr learned my addr 172.21.15.19:0/844376485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65bfff640 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebee4d0 0x56119efcee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc660dee640 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efdae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 -- 172.21.15.19:0/844376485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebee4d0 msgr2=0x56119efcee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebee4d0 0x56119efcee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 -- 172.21.15.19:0/844376485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 msgr2=0x56119efdae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efdae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 -- 172.21.15.19:0/844376485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119efc1580 con 0x56119ebf66b0 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.290+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebf66b0 0x56119efbab20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc6500a6940 tx=0x7fc6500e0f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23709f64e020708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.292+0000 7fc641ffb640 1 -- 172.21.15.19:0/844376485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6500dee60 con 0x56119ebf66b0 2024-04-25T11:04:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.292+0000 7fc641ffb640 1 -- 172.21.15.19:0/844376485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6500af3e0 con 0x56119ebf66b0 2024-04-25T11:04:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.292+0000 7fc641ffb640 1 -- 172.21.15.19:0/844376485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6500afaa0 con 0x56119ebf66b0 2024-04-25T11:04:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 -- 172.21.15.19:0/844376485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebf66b0 msgr2=0x56119efbab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebf66b0 0x56119efbab20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc6500a6940 tx=0x7fc6500e0f30 comp rx=0 tx=0).stop 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 -- 172.21.15.19:0/844376485 shutdown_connections 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efdae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebf66b0 0x56119efbab20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 --2- 172.21.15.19:0/844376485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebee4d0 0x56119efcee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 -- 172.21.15.19:0/844376485 >> 172.21.15.19:0/844376485 conn(0x56119ec0b4e0 msgr2=0x56119efba410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 -- 172.21.15.19:0/844376485 shutdown_connections 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.293+0000 7fc6697e5740 1 -- 172.21.15.19:0/844376485 wait complete. 2024-04-25T11:04:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 -- start start 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 Processor -- start 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119f035dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119f036310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efc65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1ae840 con 0x56119ebee4d0 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1afcb0 con 0x56119ec17390 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1aef10 con 0x56119ebf66b0 2024-04-25T11:04:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc6697e5740 10 client.0 _ll_register_callbacks cb 0x7fc64c005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119f035dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119f035dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35336/0 (socket says 172.21.15.19:35336) 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc65bfff640 1 -- 172.21.15.19:0/1579514336 learned_addr learned my addr 172.21.15.19:0/1579514336 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.294+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119f036310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc660dee640 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efc65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc65bfff640 1 -- 172.21.15.19:0/1579514336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 msgr2=0x56119f036310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc65bfff640 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119f036310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc65bfff640 1 -- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 msgr2=0x56119efc65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc65bfff640 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efc65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc65bfff640 1 -- 172.21.15.19:0/1579514336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119efc6bb0 con 0x56119ebee4d0 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc65bfff640 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119f035dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc64c006010 tx=0x7fc64c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5847b7a237edcd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.295+0000 7fc63b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc63b7fe640 1 -- 172.21.15.19:0/1579514336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c035030 con 0x56119ebee4d0 2024-04-25T11:04:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc63b7fe640 1 -- 172.21.15.19:0/1579514336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8557+0+0 (secure 0 0 0) 0x7fc64c0041e0 con 0x56119ebee4d0 2024-04-25T11:04:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56119eea0610 con 0x56119ebee4d0 2024-04-25T11:04:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc63b7fe640 1 -- 172.21.15.19:0/1579514336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc64c052040 con 0x56119ebee4d0 2024-04-25T11:04:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc6697e5740 10 client.17010 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56119f03d3f0 con 0x56119ebee4d0 2024-04-25T11:04:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc63b7fe640 1 -- 172.21.15.19:0/1579514336 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56119eea0610 con 0x56119ebee4d0 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.297+0000 7fc63b7fe640 1 -- 172.21.15.19:0/1579514336 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc64c055070 con 0x56119ebee4d0 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc6697e5740 10 client.17010 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc6697e5740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc6697e5740 20 client.17010 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc6697e5740 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebab3a0 0x56119f03fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc6697e5740 4 client.17010 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56119eeee240 con 0x56119ebab3a0 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebab3a0 0x56119f03fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc63b7fe640 10 client.17010 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc65b7fe640 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebab3a0 0x56119f03fb70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbeda1b0ec937b1 server_cookie=c6648d874d805a03 in_seq=0 out_seq=0 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc63b7fe640 1 -- 172.21.15.19:0/1579514336 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56119eeee240 con 0x56119ebab3a0 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc63b7fe640 10 client.17010 handle_command_reply: tid=0 2024-04-25T11:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.298+0000 7fc6697e5740 1 client.17010 shutdown 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 msgr2=0x56119f035dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119f035dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc64c006010 tx=0x7fc64c0058e0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 shutdown_connections 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x56119ebab3a0 0x56119f03fb70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56119ec17390 0x56119efc65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc63b7fe640 0 client.17010 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56119ebf66b0 0x56119f036310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 --2- 172.21.15.19:0/1579514336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119ebee4d0 0x56119f035dd0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 >> 172.21.15.19:0/1579514336 conn(0x56119ec0b4e0 msgr2=0x56119efbae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 shutdown_connections 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 1 -- 172.21.15.19:0/1579514336 wait complete. 2024-04-25T11:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc6697e5740 20 client.17010 trim_cache size 0 max 16384 2024-04-25T11:04:12.303 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6440afce0 msgr2=0x7fc6440b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc662851640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6440afce0 0x7fc6440b21f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc65401e000 tx=0x7fc654021000 comp rx=0 tx=0).stop 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 msgr2=0x7fc65c19ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.299+0000 7fc662851640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c19ad10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc65000b450 tx=0x7fc65000b910 comp rx=0 tx=0).stop 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 shutdown_connections 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fc6440afce0 0x7fc6440b21f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65c10e550 0x7fc65c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65c109ba0 0x7fc65c19ad10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 --2- 172.21.15.19:0/2553758902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65c102050 0x7fc65c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 >> 172.21.15.19:0/2553758902 conn(0x7fc65c077880 msgr2=0x7fc65c10a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 shutdown_connections 2024-04-25T11:04:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:12.300+0000 7fc662851640 1 -- 172.21.15.19:0/2553758902 wait complete. 2024-04-25T11:04:12.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:12 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/246618273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:04:12.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:12 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/246618273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:04:12.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:12 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/246618273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:04:13.311 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T11:04:13.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:04:13.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T11:04:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 -- 172.21.15.19:0/1967711743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118109f90 msgr2=0x7f01181020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 --2- 172.21.15.19:0/1967711743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118109f90 0x7f01181020b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0114009900 tx=0x7f011402a410 comp rx=0 tx=0).stop 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 -- 172.21.15.19:0/1967711743 shutdown_connections 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 --2- 172.21.15.19:0/1967711743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0118102720 0x7f0118110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 --2- 172.21.15.19:0/1967711743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118109f90 0x7f01181020b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 --2- 172.21.15.19:0/1967711743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0118109af0 0x7f0118101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 -- 172.21.15.19:0/1967711743 >> 172.21.15.19:0/1967711743 conn(0x7f0118077880 msgr2=0x7f0118077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 -- 172.21.15.19:0/1967711743 shutdown_connections 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 -- 172.21.15.19:0/1967711743 wait complete. 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 Processor -- start 2024-04-25T11:04:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.442+0000 7f01203a8640 1 -- start start 2024-04-25T11:04:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f01203a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118102720 0x7f0118102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f01203a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0118109af0 0x7f0118103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f01203a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0118109f90 0x7f01181037d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f01203a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0118113d30 con 0x7f0118102720 2024-04-25T11:04:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f01203a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0118113bb0 con 0x7f0118109af0 2024-04-25T11:04:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f01203a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0118113eb0 con 0x7f0118109f90 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118102720 0x7f0118102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118102720 0x7f0118102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51132/0 (socket says 172.21.15.19:51132) 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 -- 172.21.15.19:0/2673202418 learned_addr learned my addr 172.21.15.19:0/2673202418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e945640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0118109f90 0x7f01181037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011d943640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0118109af0 0x7f0118103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 -- 172.21.15.19:0/2673202418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0118109f90 msgr2=0x7f01181037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0118109f90 0x7f01181037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 -- 172.21.15.19:0/2673202418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0118109af0 msgr2=0x7f0118103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0118109af0 0x7f0118103290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 -- 172.21.15.19:0/2673202418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01181a8a30 con 0x7f0118102720 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e945640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0118109f90 0x7f01181037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011d943640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0118109af0 0x7f0118103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.443+0000 7f011e144640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118102720 0x7f0118102d50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f010800d950 tx=0x7f010800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.444+0000 7f010f7fe640 1 -- 172.21.15.19:0/2673202418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108017070 con 0x7f0118102720 2024-04-25T11:04:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.444+0000 7f010f7fe640 1 -- 172.21.15.19:0/2673202418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0108009c30 con 0x7f0118102720 2024-04-25T11:04:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.444+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01181a8d20 con 0x7f0118102720 2024-04-25T11:04:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.444+0000 7f010f7fe640 1 -- 172.21.15.19:0/2673202418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010800f620 con 0x7f0118102720 2024-04-25T11:04:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.444+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01181a9200 con 0x7f0118102720 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.446+0000 7f010f7fe640 1 -- 172.21.15.19:0/2673202418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0108010460 con 0x7f0118102720 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 Processor -- start 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 -- start start 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76c5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563eb762bb00 con 0x563eb7304730 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563eb75c7920 con 0x563eb7325410 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f0127256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563eb7627ac0 con 0x563eb72fc550 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f010f7fe640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f01040afce0 0x7f01040b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f011e945640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f01040afce0 0x7f01040b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51142/0 (socket says 172.21.15.19:51142) 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.447+0000 7f011d943640 1 -- 172.21.15.19:0/569955796 learned_addr learned my addr 172.21.15.19:0/569955796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011e945640 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f010f7fe640 1 -- 172.21.15.19:0/2673202418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f01080dec90 con 0x7f0118102720 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011e144640 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011d943640 1 -- 172.21.15.19:0/569955796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 msgr2=0x563eb76c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011d943640 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011d943640 1 -- 172.21.15.19:0/569955796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 msgr2=0x563eb76c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011d943640 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011d943640 1 -- 172.21.15.19:0/569955796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb74057e0 con 0x563eb7304730 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011e945640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f01040afce0 0x7f01040b21f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0110009810 tx=0x7f0110009bb0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f011d943640 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76c60a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0114005ab0 tx=0x7f0114004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5105dd1d6bd95cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f010cbfe640 1 -- 172.21.15.19:0/569955796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011403f070 con 0x563eb7304730 2024-04-25T11:04:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f010cbfe640 1 -- 172.21.15.19:0/569955796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01140054f0 con 0x563eb7304730 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.448+0000 7f010cbfe640 1 -- 172.21.15.19:0/569955796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114035030 con 0x563eb7304730 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 -- 172.21.15.19:0/569955796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 msgr2=0x563eb76c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76c60a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0114005ab0 tx=0x7f0114004170 comp rx=0 tx=0).stop 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 -- 172.21.15.19:0/569955796 shutdown_connections 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76c60a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 --2- 172.21.15.19:0/569955796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 -- 172.21.15.19:0/569955796 >> 172.21.15.19:0/569955796 conn(0x563eb7319560 msgr2=0x563eb76c1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 -- 172.21.15.19:0/569955796 shutdown_connections 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.449+0000 7f0127256740 1 -- 172.21.15.19:0/569955796 wait complete. 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 -- start start 2024-04-25T11:04:13.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 Processor -- start 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563eb76d2910 con 0x563eb7304730 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563eb75c55d0 con 0x563eb7325410 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563eb76d2a90 con 0x563eb72fc550 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f0127256740 10 client.0 _ll_register_callbacks cb 0x563eb75b3080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51148/0 (socket says 172.21.15.19:51148) 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f011d943640 1 -- 172.21.15.19:0/151315860 learned_addr learned my addr 172.21.15.19:0/151315860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.450+0000 7f011e144640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011e945640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011d943640 1 -- 172.21.15.19:0/151315860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 msgr2=0x563eb76cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011d943640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011d943640 1 -- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 msgr2=0x563eb76dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011d943640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76dbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011d943640 1 -- 172.21.15.19:0/151315860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb7465f70 con 0x563eb7304730 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011e945640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f011d943640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76d00a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f01140092a0 tx=0x7f0114004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c5ec1c2b0c5e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0101ffb640 1 -- 172.21.15.19:0/151315860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011404c020 con 0x563eb7304730 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0101ffb640 1 -- 172.21.15.19:0/151315860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f0114004f10 con 0x563eb7304730 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01140054f0 con 0x563eb7304730 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0101ffb640 1 -- 172.21.15.19:0/151315860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0114057050 con 0x563eb7304730 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0127256740 10 client.17022 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563eb72fd170 con 0x563eb7304730 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.451+0000 7f0101ffb640 1 -- 172.21.15.19:0/151315860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01140054f0 con 0x563eb7304730 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f0101ffb640 1 -- 172.21.15.19:0/151315860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f01140314a0 con 0x563eb7304730 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f0127256740 10 client.17022 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f0127256740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f0127256740 20 client.17022 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f0127256740 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72e41f0 0x563eb76e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f0127256740 4 client.17022 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eb7639710 con 0x563eb72e41f0 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.452+0000 7f011e144640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72e41f0 0x563eb76e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0101ffb640 10 client.17022 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f011e144640 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72e41f0 0x563eb76e1d20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5f2eaf72691613 server_cookie=d1bd6f3010ab1b81 in_seq=0 out_seq=0 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0101ffb640 1 -- 172.21.15.19:0/151315860 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563eb7639710 con 0x563eb72e41f0 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0101ffb640 10 client.17022 handle_command_reply: tid=0 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 client.17022 shutdown 2024-04-25T11:04:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 msgr2=0x563eb76d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76d00a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f01140092a0 tx=0x7f0114004610 comp rx=0 tx=0).stop 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 shutdown_connections 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72e41f0 0x563eb76e1d20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb7325410 0x563eb76dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76d00a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0101ffb640 0 client.17022 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.453+0000 7f0127256740 1 --2- 172.21.15.19:0/151315860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb72fc550 0x563eb76cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.454+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 >> 172.21.15.19:0/151315860 conn(0x563eb7319560 msgr2=0x563eb76c7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.454+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 shutdown_connections 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.454+0000 7f0127256740 1 -- 172.21.15.19:0/151315860 wait complete. 2024-04-25T11:04:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.454+0000 7f0127256740 20 client.17022 trim_cache size 0 max 16384 2024-04-25T11:04:13.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 Processor -- start 2024-04-25T11:04:13.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 -- start start 2024-04-25T11:04:13.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 0x563eb76dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01181afba0 con 0x563eb7304730 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01181b2400 con 0x563eb72fc550 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f0127256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01181b01f0 con 0x563eb7325410 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51150/0 (socket says 172.21.15.19:51150) 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.472+0000 7f011d943640 1 -- 172.21.15.19:0/2244142694 learned_addr learned my addr 172.21.15.19:0/2244142694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011e144640 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011d943640 1 -- 172.21.15.19:0/2244142694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 msgr2=0x563eb76dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011d943640 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 0x563eb76dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011d943640 1 -- 172.21.15.19:0/2244142694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 msgr2=0x563eb76c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011d943640 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011d943640 1 -- 172.21.15.19:0/2244142694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb76c6110 con 0x563eb7304730 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011e144640 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f011d943640 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76dbd30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f011402a9d0 tx=0x7f0114031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956e3f551c493264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f0103fff640 1 -- 172.21.15.19:0/2244142694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011404c070 con 0x563eb7304730 2024-04-25T11:04:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f0103fff640 1 -- 172.21.15.19:0/2244142694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0114031b30 con 0x563eb7304730 2024-04-25T11:04:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.473+0000 7f0103fff640 1 -- 172.21.15.19:0/2244142694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011403f070 con 0x563eb7304730 2024-04-25T11:04:13.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- 172.21.15.19:0/2244142694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 msgr2=0x563eb76dbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76dbd30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f011402a9d0 tx=0x7f0114031400 comp rx=0 tx=0).stop 2024-04-25T11:04:13.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- 172.21.15.19:0/2244142694 shutdown_connections 2024-04-25T11:04:13.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 0x563eb76dc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76dbd30 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 --2- 172.21.15.19:0/2244142694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- 172.21.15.19:0/2244142694 >> 172.21.15.19:0/2244142694 conn(0x563eb7319560 msgr2=0x563eb76c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- 172.21.15.19:0/2244142694 shutdown_connections 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- 172.21.15.19:0/2244142694 wait complete. 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- start start 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 Processor -- start 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76e2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76e2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 0x563eb76e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01181b08c0 con 0x563eb7304730 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01181b1d30 con 0x563eb72fc550 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01181b0f90 con 0x563eb7325410 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f0127256740 10 client.0 _ll_register_callbacks cb 0x7f011402ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.474+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76e2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76e2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51158/0 (socket says 172.21.15.19:51158) 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 -- 172.21.15.19:0/3675874320 learned_addr learned my addr 172.21.15.19:0/3675874320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011e945640 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 0x563eb76e33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011e144640 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76e2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 -- 172.21.15.19:0/3675874320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 msgr2=0x563eb76e33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 0x563eb76e33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 -- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 msgr2=0x563eb76e2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76e2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 -- 172.21.15.19:0/3675874320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb7748e50 con 0x563eb7304730 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f011d943640 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76e2e70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f011403d040 tx=0x7f011402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e38ebc37d1aea51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f0101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f0101ffb640 1 -- 172.21.15.19:0/3675874320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011404c040 con 0x563eb7304730 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f0101ffb640 1 -- 172.21.15.19:0/3675874320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f01140037f0 con 0x563eb7304730 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0114031b30 con 0x563eb7304730 2024-04-25T11:04:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.475+0000 7f0101ffb640 1 -- 172.21.15.19:0/3675874320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f011403f040 con 0x563eb7304730 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 10 client.17028 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563eb7749fe0 con 0x563eb7304730 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0101ffb640 1 -- 172.21.15.19:0/3675874320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0114031b30 con 0x563eb7304730 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0101ffb640 1 -- 172.21.15.19:0/3675874320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f011403bcb0 con 0x563eb7304730 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 10 client.17028 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 20 client.17028 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72b9630 0x563eb774c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 4 client.17028 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563eb76d0430 con 0x563eb72b9630 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.476+0000 7f011e144640 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72b9630 0x563eb774c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.477+0000 7f0101ffb640 10 client.17028 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.477+0000 7f011e144640 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72b9630 0x563eb774c560 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79077745ce3310a0 server_cookie=da8eb646348c837 in_seq=0 out_seq=0 2024-04-25T11:04:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0101ffb640 1 -- 172.21.15.19:0/3675874320 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563eb76d0430 con 0x563eb72b9630 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0101ffb640 10 client.17028 handle_command_reply: tid=0 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 client.17028 shutdown 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 msgr2=0x563eb76e2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76e2e70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f011403d040 tx=0x7f011402a410 comp rx=0 tx=0).stop 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 shutdown_connections 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x563eb72b9630 0x563eb774c560 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eb7325410 0x563eb76e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eb7304730 0x563eb76e2e70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 --2- 172.21.15.19:0/3675874320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eb72fc550 0x563eb76e2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 >> 172.21.15.19:0/3675874320 conn(0x563eb7319560 msgr2=0x563eb76cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0101ffb640 0 client.17028 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 shutdown_connections 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 1 -- 172.21.15.19:0/3675874320 wait complete. 2024-04-25T11:04:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.478+0000 7f0127256740 20 client.17028 trim_cache size 0 max 16384 2024-04-25T11:04:13.484 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:13.484 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:04:13.484 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "b3618cec-9d69-4808-9e0b-dda7276c6b4d", 2024-04-25T11:04:13.484 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:04:13.484 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f01040afce0 msgr2=0x7f01040b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f01040afce0 0x7f01040b21f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0110009810 tx=0x7f0110009bb0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118102720 msgr2=0x7f0118102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118102720 0x7f0118102d50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f010800d950 tx=0x7f010800de10 comp rx=0 tx=0).stop 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 shutdown_connections 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f01040afce0 0x7f01040b21f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0118109f90 0x7f01181037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0118109af0 0x7f0118103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 --2- 172.21.15.19:0/2673202418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0118102720 0x7f0118102d50 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 >> 172.21.15.19:0/2673202418 conn(0x7f0118077880 msgr2=0x7f01181051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 shutdown_connections 2024-04-25T11:04:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.479+0000 7f01203a8640 1 -- 172.21.15.19:0/2673202418 wait complete. 2024-04-25T11:04:13.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:04:13.490 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:04:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:13 smithi112 ceph-mon[32799]: pgmap v301: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 39 KiB/s wr, 7 op/s 2024-04-25T11:04:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:13 smithi112 ceph-mon[32799]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T11:04:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 -- 172.21.15.19:0/2150362234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438104270 msgr2=0x7ff438110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2150362234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438104270 0x7ff438110640 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff428009900 tx=0x7ff42802a410 comp rx=0 tx=0).stop 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 -- 172.21.15.19:0/2150362234 shutdown_connections 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2150362234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438104270 0x7ff438110640 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2150362234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438109fc0 0x7ff438103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2150362234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff438109b20 0x7ff4381036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 -- 172.21.15.19:0/2150362234 >> 172.21.15.19:0/2150362234 conn(0x7ff4380ff1e0 msgr2=0x7ff438101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 -- 172.21.15.19:0/2150362234 shutdown_connections 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.604+0000 7ff43eb38640 1 -- 172.21.15.19:0/2150362234 wait complete. 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 Processor -- start 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 -- start start 2024-04-25T11:04:13.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff438104270 0x7ff43819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438109b20 0x7ff43819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438109fc0 0x7ff43819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113d30 con 0x7ff438109fc0 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113bb0 con 0x7ff438109b20 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43eb38640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113eb0 con 0x7ff438104270 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438109fc0 0x7ff43819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438109fc0 0x7ff43819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51170/0 (socket says 172.21.15.19:51170) 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 -- 172.21.15.19:0/2365556786 learned_addr learned my addr 172.21.15.19:0/2365556786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff437fff640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438109b20 0x7ff43819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff438104270 0x7ff43819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 -- 172.21.15.19:0/2365556786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff438104270 msgr2=0x7ff43819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff438104270 0x7ff43819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 -- 172.21.15.19:0/2365556786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438109b20 msgr2=0x7ff43819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438109b20 0x7ff43819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.605+0000 7ff43d0d5640 1 -- 172.21.15.19:0/2365556786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43819f590 con 0x7ff438109fc0 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.606+0000 7ff437fff640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438109b20 0x7ff43819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.606+0000 7ff43d0d5640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438109fc0 0x7ff43819ef50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff428004770 tx=0x7ff428004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.606+0000 7ff435ffb640 1 -- 172.21.15.19:0/2365556786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42803f030 con 0x7ff438109fc0 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.606+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43819f820 con 0x7ff438109fc0 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.606+0000 7ff435ffb640 1 -- 172.21.15.19:0/2365556786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff428003aa0 con 0x7ff438109fc0 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.606+0000 7ff435ffb640 1 -- 172.21.15.19:0/2365556786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428035070 con 0x7ff438109fc0 2024-04-25T11:04:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.606+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43819fd60 con 0x7ff438109fc0 2024-04-25T11:04:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.608+0000 7ff4459ed740 1 Processor -- start 2024-04-25T11:04:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff435ffb640 1 -- 172.21.15.19:0/2365556786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff428042030 con 0x7ff438109fc0 2024-04-25T11:04:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff4459ed740 1 -- start start 2024-04-25T11:04:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff4459ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9d2219d80 con 0x55c9d1f13390 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff4459ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c9d21d38b0 con 0x55c9d1ef26b0 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff4459ed740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c9d2215d40 con 0x55c9d1eea4d0 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff437fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff437fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51178/0 (socket says 172.21.15.19:51178) 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff437fff640 1 -- 172.21.15.19:0/835826679 learned_addr learned my addr 172.21.15.19:0/835826679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.609+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff435ffb640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4240afc60 0x7ff4240b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff437fff640 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff437fff640 1 -- 172.21.15.19:0/835826679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 msgr2=0x55c9d22b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff437fff640 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff437fff640 1 -- 172.21.15.19:0/835826679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 msgr2=0x55c9d22c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff437fff640 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff43d0d5640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4240afc60 0x7ff4240b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff437fff640 1 -- 172.21.15.19:0/835826679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d21a4a40 con 0x55c9d1f13390 2024-04-25T11:04:13.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff435ffb640 1 -- 172.21.15.19:0/2365556786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7ff4280ff920 con 0x7ff438109fc0 2024-04-25T11:04:13.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:13 smithi019 ceph-mon[29209]: pgmap v301: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 39 KiB/s wr, 7 op/s 2024-04-25T11:04:13.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:13 smithi019 ceph-mon[29209]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T11:04:13.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff437fff640 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22b3a60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff42000bfc0 tx=0x7ff42000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef46d2ca6891b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff42effd640 1 -- 172.21.15.19:0/835826679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420019030 con 0x55c9d1f13390 2024-04-25T11:04:13.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff43d0d5640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4240afc60 0x7ff4240b2170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff4280c4cd0 tx=0x7ff42808e320 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff42effd640 1 -- 172.21.15.19:0/835826679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4200035f0 con 0x55c9d1f13390 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.610+0000 7ff42effd640 1 -- 172.21.15.19:0/835826679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420011720 con 0x55c9d1f13390 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 -- 172.21.15.19:0/835826679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 msgr2=0x55c9d22b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22b3a60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff42000bfc0 tx=0x7ff42000b650 comp rx=0 tx=0).stop 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 -- 172.21.15.19:0/835826679 shutdown_connections 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 --2- 172.21.15.19:0/835826679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22b3a60 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 -- 172.21.15.19:0/835826679 >> 172.21.15.19:0/835826679 conn(0x55c9d1f074e0 msgr2=0x55c9d22b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 -- 172.21.15.19:0/835826679 shutdown_connections 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.611+0000 7ff4459ed740 1 -- 172.21.15.19:0/835826679 wait complete. 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 -- start start 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 Processor -- start 2024-04-25T11:04:13.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9d22c0fe0 con 0x55c9d1f13390 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c9d21a0d10 con 0x55c9d1ef26b0 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c9d22c1160 con 0x55c9d1eea4d0 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff4459ed740 10 client.0 _ll_register_callbacks cb 0x55c9d21cd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff437fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff437fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22c44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51184/0 (socket says 172.21.15.19:51184) 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff437fff640 1 -- 172.21.15.19:0/3235238032 learned_addr learned my addr 172.21.15.19:0/3235238032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.612+0000 7ff437fff640 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff437fff640 1 -- 172.21.15.19:0/3235238032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 msgr2=0x55c9d22be640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff437fff640 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22be640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff437fff640 1 -- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 msgr2=0x55c9d22beb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff437fff640 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22beb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff437fff640 1 -- 172.21.15.19:0/3235238032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d1ff57d0 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff437fff640 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22c44a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff420003e50 tx=0x7ff420003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eccf7b50a023eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff417fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff417fff640 1 -- 172.21.15.19:0/3235238032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420019050 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff417fff640 1 -- 172.21.15.19:0/3235238032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7ff420004000 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4200035f0 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff417fff640 1 -- 172.21.15.19:0/3235238032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff420034050 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff4459ed740 10 client.17040 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c9d1f2b3f0 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.613+0000 7ff417fff640 1 -- 172.21.15.19:0/3235238032 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4200035f0 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff417fff640 1 -- 172.21.15.19:0/3235238032 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff42002c020 con 0x55c9d1f13390 2024-04-25T11:04:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff4459ed740 10 client.17040 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff4459ed740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff4459ed740 20 client.17040 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ed1f60 0x55c9d22d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff4459ed740 4 client.17040 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9d2227990 con 0x55c9d1ed1f60 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ed1f60 0x55c9d22d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff417fff640 10 client.17040 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.614+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ed1f60 0x55c9d22d06a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8572e30a5d9db91 server_cookie=70b62614ca257cee in_seq=0 out_seq=0 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff417fff640 1 -- 172.21.15.19:0/3235238032 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9d2227990 con 0x55c9d1ed1f60 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff417fff640 10 client.17040 handle_command_reply: tid=0 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 client.17040 shutdown 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 msgr2=0x55c9d22c44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22c44a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff420003e50 tx=0x7ff420003f50 comp rx=0 tx=0).stop 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 shutdown_connections 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ed1f60 0x55c9d22d06a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1f13390 0x55c9d22c44a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1ef26b0 0x55c9d22beb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff417fff640 0 client.17040 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3235238032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22be640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.615+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 >> 172.21.15.19:0/3235238032 conn(0x55c9d1f074e0 msgr2=0x55c9d22ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.616+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 shutdown_connections 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.616+0000 7ff4459ed740 1 -- 172.21.15.19:0/3235238032 wait complete. 2024-04-25T11:04:13.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.616+0000 7ff4459ed740 20 client.17040 trim_cache size 0 max 16384 2024-04-25T11:04:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 Processor -- start 2024-04-25T11:04:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 -- start start 2024-04-25T11:04:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1ef26b0 0x55c9d22d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381ad960 con 0x55c9d1ef26b0 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381b01c0 con 0x55c9d1f13390 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff4459ed740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381adfb0 con 0x55c9d1eea4d0 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff43c8d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1ef26b0 0x55c9d22d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff43c8d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1ef26b0 0x55c9d22d6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51198/0 (socket says 172.21.15.19:51198) 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff43c8d4640 1 -- 172.21.15.19:0/3177677411 learned_addr learned my addr 172.21.15.19:0/3177677411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff437fff640 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.634+0000 7ff437fff640 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff43c8d4640 1 -- 172.21.15.19:0/3177677411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 msgr2=0x55c9d22d5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22d5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff43c8d4640 1 -- 172.21.15.19:0/3177677411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 msgr2=0x55c9d22ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff43c8d4640 1 -- 172.21.15.19:0/3177677411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d1eeb0f0 con 0x55c9d1ef26b0 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff437fff640 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1ef26b0 0x55c9d22d6270 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff438104800 tx=0x7ff430013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e18be257c75e084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff42e7fc640 1 -- 172.21.15.19:0/3177677411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430004d20 con 0x55c9d1ef26b0 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff42e7fc640 1 -- 172.21.15.19:0/3177677411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff430023030 con 0x55c9d1ef26b0 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.635+0000 7ff42e7fc640 1 -- 172.21.15.19:0/3177677411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430005020 con 0x55c9d1ef26b0 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 -- 172.21.15.19:0/3177677411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1ef26b0 msgr2=0x55c9d22d6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1ef26b0 0x55c9d22d6270 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff438104800 tx=0x7ff430013ce0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 -- 172.21.15.19:0/3177677411 shutdown_connections 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1ef26b0 0x55c9d22d6270 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 --2- 172.21.15.19:0/3177677411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1eea4d0 0x55c9d22d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 -- 172.21.15.19:0/3177677411 >> 172.21.15.19:0/3177677411 conn(0x55c9d1f074e0 msgr2=0x55c9d22bcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 -- 172.21.15.19:0/3177677411 shutdown_connections 2024-04-25T11:04:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 -- 172.21.15.19:0/3177677411 wait complete. 2024-04-25T11:04:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 -- start start 2024-04-25T11:04:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.636+0000 7ff4459ed740 1 Processor -- start 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1eea4d0 0x55c9d22ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381ae680 con 0x55c9d1eea4d0 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381afaf0 con 0x55c9d1f13390 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381aed50 con 0x55c9d1ef26b0 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 10 client.0 _ll_register_callbacks cb 0x7ff430009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1eea4d0 0x55c9d22ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1eea4d0 0x55c9d22ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51214/0 (socket says 172.21.15.19:51214) 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 -- 172.21.15.19:0/938240107 learned_addr learned my addr 172.21.15.19:0/938240107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:13.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 -- 172.21.15.19:0/938240107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1ef26b0 msgr2=0x55c9d22c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 -- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 msgr2=0x55c9d22c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 -- 172.21.15.19:0/938240107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d21da4b0 con 0x55c9d1eea4d0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff437fff640 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1eea4d0 0x55c9d22ca7a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff420003e50 tx=0x7ff42000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d03dcfd7d3551d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff417fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff417fff640 1 -- 172.21.15.19:0/938240107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420019040 con 0x55c9d1eea4d0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff417fff640 1 -- 172.21.15.19:0/938240107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7ff420003800 con 0x55c9d1eea4d0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.637+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff430023030 con 0x55c9d1eea4d0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff417fff640 1 -- 172.21.15.19:0/938240107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff420046050 con 0x55c9d1eea4d0 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 10 client.17046 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c9d23376d0 con 0x55c9d1eea4d0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff417fff640 1 -- 172.21.15.19:0/938240107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff430023030 con 0x55c9d1eea4d0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff417fff640 1 -- 172.21.15.19:0/938240107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff420041030 con 0x55c9d1eea4d0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 10 client.17046 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 20 client.17046 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ea73a0 0x55c9d2339e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 4 client.17046 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9d2221800 con 0x55c9d1ea73a0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.638+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ea73a0 0x55c9d2339e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff417fff640 10 client.17046 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff43c8d4640 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ea73a0 0x55c9d2339e00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424dee2d8b696c43 server_cookie=a894da8ce6df8aa in_seq=0 out_seq=0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff417fff640 1 -- 172.21.15.19:0/938240107 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c9d2221800 con 0x55c9d1ea73a0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff417fff640 10 client.17046 handle_command_reply: tid=0 2024-04-25T11:04:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 client.17046 shutdown 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1eea4d0 msgr2=0x55c9d22ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1eea4d0 0x55c9d22ca7a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff420003e50 tx=0x7ff42000b420 comp rx=0 tx=0).stop 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 shutdown_connections 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55c9d1ea73a0 0x55c9d2339e00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c9d1f13390 0x55c9d22c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9d1ef26b0 0x55c9d22c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff417fff640 0 client.17046 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 --2- 172.21.15.19:0/938240107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d1eea4d0 0x55c9d22ca7a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 >> 172.21.15.19:0/938240107 conn(0x55c9d1f074e0 msgr2=0x55c9d22bd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 shutdown_connections 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 1 -- 172.21.15.19:0/938240107 wait complete. 2024-04-25T11:04:13.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.639+0000 7ff4459ed740 20 client.17046 trim_cache size 0 max 16384 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: "b3618cec-9d69-4808-9e0b-dda7276c6b4d": { 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "b3618cec-9d69-4808-9e0b-dda7276c6b4d", 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:13.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4240afc60 msgr2=0x7ff4240b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4240afc60 0x7ff4240b2170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff4280c4cd0 tx=0x7ff42808e320 comp rx=0 tx=0).stop 2024-04-25T11:04:13.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438109fc0 msgr2=0x7ff43819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:13.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438109fc0 0x7ff43819ef50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff428004770 tx=0x7ff428004270 comp rx=0 tx=0).stop 2024-04-25T11:04:13.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 shutdown_connections 2024-04-25T11:04:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ff4240afc60 0x7ff4240b2170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff438109fc0 0x7ff43819ef50 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438109b20 0x7ff43819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 --2- 172.21.15.19:0/2365556786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff438104270 0x7ff43819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 >> 172.21.15.19:0/2365556786 conn(0x7ff4380ff1e0 msgr2=0x7ff438101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 shutdown_connections 2024-04-25T11:04:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:13.640+0000 7ff43eb38640 1 -- 172.21.15.19:0/2365556786 wait complete. 2024-04-25T11:04:13.651 INFO:tasks.cephfs.filesystem:scrub status for tag:b3618cec-9d69-4808-9e0b-dda7276c6b4d - {'path': '/', 'tag': 'b3618cec-9d69-4808-9e0b-dda7276c6b4d', 'options': 'recursive,force'} 2024-04-25T11:04:13.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:13 smithi169 ceph-mon[33049]: pgmap v301: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 39 KiB/s wr, 7 op/s 2024-04-25T11:04:13.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:13 smithi169 ceph-mon[33049]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T11:04:14.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:14 smithi112 ceph-mon[32799]: scrub queued for path: / 2024-04-25T11:04:14.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:14 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:04:14.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:14 smithi112 ceph-mon[32799]: scrub summary: active paths [/] 2024-04-25T11:04:14.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:14 smithi112 ceph-mon[32799]: scrub summary: idle+waiting paths [/] 2024-04-25T11:04:14.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:14 smithi019 ceph-mon[29209]: scrub queued for path: / 2024-04-25T11:04:14.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:14 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:04:14.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:14 smithi019 ceph-mon[29209]: scrub summary: active paths [/] 2024-04-25T11:04:14.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:14 smithi019 ceph-mon[29209]: scrub summary: idle+waiting paths [/] 2024-04-25T11:04:14.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:14 smithi169 ceph-mon[33049]: scrub queued for path: / 2024-04-25T11:04:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:14 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:04:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:14 smithi169 ceph-mon[33049]: scrub summary: active paths [/] 2024-04-25T11:04:14.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:14 smithi169 ceph-mon[33049]: scrub summary: idle+waiting paths [/] 2024-04-25T11:04:15.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:15 smithi112 ceph-mon[32799]: pgmap v303: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:04:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:15 smithi019 ceph-mon[29209]: pgmap v303: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:04:15.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:15 smithi169 ceph-mon[33049]: pgmap v303: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T11:04:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:17 smithi112 ceph-mon[32799]: pgmap v304: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 232 B/s wr, 4 op/s 2024-04-25T11:04:17.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:17 smithi112 ceph-mon[32799]: scrub summary: idle 2024-04-25T11:04:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:17 smithi019 ceph-mon[29209]: pgmap v304: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 232 B/s wr, 4 op/s 2024-04-25T11:04:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:17 smithi019 ceph-mon[29209]: scrub summary: idle 2024-04-25T11:04:17.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:17 smithi169 ceph-mon[33049]: pgmap v304: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 232 B/s wr, 4 op/s 2024-04-25T11:04:17.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:17 smithi169 ceph-mon[33049]: scrub summary: idle 2024-04-25T11:04:19.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:18 smithi169 ceph-mon[33049]: pgmap v305: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:04:19.058 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:18 smithi112 ceph-mon[32799]: pgmap v305: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:04:19.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:18 smithi019 ceph-mon[29209]: pgmap v305: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:04:21.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:21 smithi169 ceph-mon[33049]: pgmap v306: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:04:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:21 smithi112 ceph-mon[32799]: pgmap v306: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:04:21.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:21 smithi019 ceph-mon[29209]: pgmap v306: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-25T11:04:23.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:23 smithi169 ceph-mon[33049]: pgmap v307: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 102 B/s wr, 3 op/s 2024-04-25T11:04:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:23 smithi112 ceph-mon[32799]: pgmap v307: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 102 B/s wr, 3 op/s 2024-04-25T11:04:23.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:23 smithi019 ceph-mon[29209]: pgmap v307: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 102 B/s wr, 3 op/s 2024-04-25T11:04:25.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:25 smithi169 ceph-mon[33049]: pgmap v308: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 89 B/s wr, 2 op/s 2024-04-25T11:04:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:25 smithi112 ceph-mon[32799]: pgmap v308: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 89 B/s wr, 2 op/s 2024-04-25T11:04:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:25 smithi019 ceph-mon[29209]: pgmap v308: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 89 B/s wr, 2 op/s 2024-04-25T11:04:27.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:27 smithi169 ceph-mon[33049]: pgmap v309: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T11:04:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:27 smithi112 ceph-mon[32799]: pgmap v309: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T11:04:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:27 smithi019 ceph-mon[29209]: pgmap v309: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T11:04:29.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:29 smithi169 ceph-mon[33049]: pgmap v310: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:29.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:29 smithi112 ceph-mon[32799]: pgmap v310: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:29.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:29 smithi019 ceph-mon[29209]: pgmap v310: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:31.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:31 smithi169 ceph-mon[33049]: pgmap v311: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:31.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:31 smithi112 ceph-mon[32799]: pgmap v311: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:31.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:31 smithi019 ceph-mon[29209]: pgmap v311: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:33.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:33 smithi169 ceph-mon[33049]: pgmap v312: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:33.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:33 smithi112 ceph-mon[32799]: pgmap v312: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:33.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:33 smithi019 ceph-mon[29209]: pgmap v312: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:35.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:35 smithi169 ceph-mon[33049]: pgmap v313: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:35.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:35 smithi112 ceph-mon[32799]: pgmap v313: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:35.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:35 smithi019 ceph-mon[29209]: pgmap v313: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:36.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 -- 172.21.15.19:0/1355966727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108ef0 msgr2=0x7f77e4102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/1355966727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108ef0 0x7f77e4102b40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009900 tx=0x7f77d402a410 comp rx=0 tx=0).stop 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 -- 172.21.15.19:0/1355966727 shutdown_connections 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/1355966727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e41031b0 0x7f77e410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/1355966727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108ef0 0x7f77e4102b40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/1355966727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e4108a50 0x7f77e4102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 -- 172.21.15.19:0/1355966727 >> 172.21.15.19:0/1355966727 conn(0x7f77e40fe260 msgr2=0x7f77e4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 -- 172.21.15.19:0/1355966727 shutdown_connections 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.972+0000 7f77eb9f3640 1 -- 172.21.15.19:0/1355966727 wait complete. 2024-04-25T11:04:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 Processor -- start 2024-04-25T11:04:36.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 -- start start 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e41031b0 0x7f77e41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108a50 0x7f77e4199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e4108ef0 0x7f77e419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4112c70 con 0x7f77e4108a50 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4112af0 con 0x7f77e41031b0 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77eb9f3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4112df0 con 0x7f77e4108ef0 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108a50 0x7f77e4199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108a50 0x7f77e4199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39172/0 (socket says 172.21.15.19:39172) 2024-04-25T11:04:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 -- 172.21.15.19:0/2425301508 learned_addr learned my addr 172.21.15.19:0/2425301508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e978f640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e41031b0 0x7f77e41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e9f90640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e4108ef0 0x7f77e419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 -- 172.21.15.19:0/2425301508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e4108ef0 msgr2=0x7f77e419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e4108ef0 0x7f77e419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 -- 172.21.15.19:0/2425301508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e41031b0 msgr2=0x7f77e41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e41031b0 0x7f77e41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e8f8e640 1 -- 172.21.15.19:0/2425301508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e419e530 con 0x7f77e4108a50 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e978f640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e41031b0 0x7f77e41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.973+0000 7f77e9f90640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e4108ef0 0x7f77e419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.974+0000 7f77e8f8e640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108a50 0x7f77e4199c30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006010 tx=0x7f77d4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.974+0000 7f77da7fc640 1 -- 172.21.15.19:0/2425301508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4046070 con 0x7f77e4108a50 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.974+0000 7f77da7fc640 1 -- 172.21.15.19:0/2425301508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d40044a0 con 0x7f77e4108a50 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.974+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e419e7c0 con 0x7f77e4108a50 2024-04-25T11:04:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.974+0000 7f77da7fc640 1 -- 172.21.15.19:0/2425301508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4035050 con 0x7f77e4108a50 2024-04-25T11:04:36.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.974+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e419ed00 con 0x7f77e4108a50 2024-04-25T11:04:36.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.975+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e4050740 con 0x7f77e4108a50 2024-04-25T11:04:36.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.976+0000 7f77da7fc640 1 -- 172.21.15.19:0/2425301508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f77d4049070 con 0x7f77e4108a50 2024-04-25T11:04:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.977+0000 7f77da7fc640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f77d00afcd0 0x7f77d00b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.977+0000 7f77e978f640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f77d00afcd0 0x7f77d00b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.977+0000 7f77da7fc640 1 -- 172.21.15.19:0/2425301508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f77d4100c70 con 0x7f77e4108a50 2024-04-25T11:04:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.978+0000 7f77e978f640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f77d00afcd0 0x7f77d00b21e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77e406c550 tx=0x7f77e0005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:36.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:36.981+0000 7f77da7fc640 1 -- 172.21.15.19:0/2425301508 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77d4050080 con 0x7f77e4108a50 2024-04-25T11:04:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.193+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f77e40fed80 con 0x7f77d00afcd0 2024-04-25T11:04:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.201+0000 7f77da7fc640 1 -- 172.21.15.19:0/2425301508 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f77e40fed80 con 0x7f77d00afcd0 2024-04-25T11:04:37.203 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T10:54:35", "first": "2024-04-25T10:55:00", "last": "2024-04-25T11:04:00", "last_pruned": "2024-04-25T11:04:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f77d00afcd0 msgr2=0x7f77d00b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f77d00afcd0 0x7f77d00b21e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77e406c550 tx=0x7f77e0005f90 comp rx=0 tx=0).stop 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108a50 msgr2=0x7f77e4199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108a50 0x7f77e4199c30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006010 tx=0x7f77d4004060 comp rx=0 tx=0).stop 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 shutdown_connections 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f77d00afcd0 0x7f77d00b21e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e4108ef0 0x7f77e419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e4108a50 0x7f77e4199c30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 --2- 172.21.15.19:0/2425301508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e41031b0 0x7f77e41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.203+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 >> 172.21.15.19:0/2425301508 conn(0x7f77e40fe260 msgr2=0x7f77e40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.204+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 shutdown_connections 2024-04-25T11:04:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.204+0000 7f77eb9f3640 1 -- 172.21.15.19:0/2425301508 wait complete. 2024-04-25T11:04:37.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T11:04:37.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 -- 172.21.15.19:0/637494828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8109ff0 msgr2=0x7ffbe8103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:37.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 --2- 172.21.15.19:0/637494828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8109ff0 0x7ffbe8103c30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009900 tx=0x7ffbd802a410 comp rx=0 tx=0).stop 2024-04-25T11:04:37.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 -- 172.21.15.19:0/637494828 shutdown_connections 2024-04-25T11:04:37.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 --2- 172.21.15.19:0/637494828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe81042a0 0x7ffbe8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 --2- 172.21.15.19:0/637494828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8109ff0 0x7ffbe8103c30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 --2- 172.21.15.19:0/637494828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8109b50 0x7ffbe81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 -- 172.21.15.19:0/637494828 >> 172.21.15.19:0/637494828 conn(0x7ffbe80ff230 msgr2=0x7ffbe8101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 -- 172.21.15.19:0/637494828 shutdown_connections 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.328+0000 7ffbefb46640 1 -- 172.21.15.19:0/637494828 wait complete. 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 Processor -- start 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 -- start start 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe81042a0 0x7ffbe819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8109b50 0x7ffbe819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe8109ff0 0x7ffbe819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8113d40 con 0x7ffbe81042a0 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8113bc0 con 0x7ffbe8109b50 2024-04-25T11:04:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbefb46640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8113ec0 con 0x7ffbe8109ff0 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbed8e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe81042a0 0x7ffbe819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbed8e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe81042a0 0x7ffbe819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39188/0 (socket says 172.21.15.19:39188) 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.329+0000 7ffbed8e2640 1 -- 172.21.15.19:0/280419878 learned_addr learned my addr 172.21.15.19:0/280419878 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbee0e3640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe8109ff0 0x7ffbe819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed0e1640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8109b50 0x7ffbe819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed8e2640 1 -- 172.21.15.19:0/280419878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe8109ff0 msgr2=0x7ffbe819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed8e2640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe8109ff0 0x7ffbe819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed8e2640 1 -- 172.21.15.19:0/280419878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8109b50 msgr2=0x7ffbe819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed8e2640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8109b50 0x7ffbe819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed8e2640 1 -- 172.21.15.19:0/280419878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe819f660 con 0x7ffbe81042a0 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed0e1640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8109b50 0x7ffbe819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbed8e2640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe81042a0 0x7ffbe819a800 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400d770 tx=0x7ffbe400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbdeffd640 1 -- 172.21.15.19:0/280419878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4017070 con 0x7ffbe81042a0 2024-04-25T11:04:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbdeffd640 1 -- 172.21.15.19:0/280419878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbe4009c30 con 0x7ffbe81042a0 2024-04-25T11:04:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe819f950 con 0x7ffbe81042a0 2024-04-25T11:04:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbdeffd640 1 -- 172.21.15.19:0/280419878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400f620 con 0x7ffbe81042a0 2024-04-25T11:04:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.330+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81a72d0 con 0x7ffbe81042a0 2024-04-25T11:04:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.331+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe8050740 con 0x7ffbe81042a0 2024-04-25T11:04:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.333+0000 7ffbdeffd640 1 -- 172.21.15.19:0/280419878 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ffbe401a070 con 0x7ffbe81042a0 2024-04-25T11:04:37.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.334+0000 7ffbdeffd640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffbd40afc50 0x7ffbd40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:37.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.334+0000 7ffbed0e1640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffbd40afc50 0x7ffbd40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:37.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.334+0000 7ffbdeffd640 1 -- 172.21.15.19:0/280419878 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7ffbe40deb50 con 0x7ffbe81042a0 2024-04-25T11:04:37.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.334+0000 7ffbed0e1640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffbd40afc50 0x7ffbd40b2160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:37.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.337+0000 7ffbdeffd640 1 -- 172.21.15.19:0/280419878 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbe4009de0 con 0x7ffbe81042a0 2024-04-25T11:04:37.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:37 smithi169 ceph-mon[33049]: pgmap v314: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.548+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 --> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ffbe810fe20 con 0x7ffbd40afc50 2024-04-25T11:04:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.555+0000 7ffbdeffd640 1 -- 172.21.15.19:0/280419878 <== mgr.15210 v2:172.21.15.19:6800/2425110073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ffbe810fe20 con 0x7ffbd40afc50 2024-04-25T11:04:37.556 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-25T11:04:37.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:37 smithi112 ceph-mon[32799]: pgmap v314: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffbd40afc50 msgr2=0x7ffbd40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffbd40afc50 0x7ffbd40b2160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8031400 comp rx=0 tx=0).stop 2024-04-25T11:04:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe81042a0 msgr2=0x7ffbe819a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe81042a0 0x7ffbe819a800 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400d770 tx=0x7ffbe400dc30 comp rx=0 tx=0).stop 2024-04-25T11:04:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 shutdown_connections 2024-04-25T11:04:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7ffbd40afc50 0x7ffbd40b2160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe8109ff0 0x7ffbe819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8109b50 0x7ffbe819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 --2- 172.21.15.19:0/280419878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe81042a0 0x7ffbe819a800 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.557+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 >> 172.21.15.19:0/280419878 conn(0x7ffbe80ff230 msgr2=0x7ffbe810f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.558+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 shutdown_connections 2024-04-25T11:04:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:37.558+0000 7ffbefb46640 1 -- 172.21.15.19:0/280419878 wait complete. 2024-04-25T11:04:37.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:37 smithi019 ceph-mon[29209]: pgmap v314: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:37.570 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-25T11:04:37.582 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-25T11:04:38.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:38 smithi169 ceph-mon[33049]: from='client.17052 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:04:38.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:38 smithi112 ceph-mon[32799]: from='client.17052 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:04:38.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:38 smithi019 ceph-mon[29209]: from='client.17052 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:04:39.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:39 smithi169 ceph-mon[33049]: from='client.17058 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:04:39.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:39 smithi169 ceph-mon[33049]: pgmap v315: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:39.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:39 smithi112 ceph-mon[32799]: from='client.17058 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:04:39.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:39 smithi112 ceph-mon[32799]: pgmap v315: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:39.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:39 smithi019 ceph-mon[29209]: from='client.17058 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:04:39.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:39 smithi019 ceph-mon[29209]: pgmap v315: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:41.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:41 smithi169 ceph-mon[33049]: pgmap v316: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:41.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:41 smithi112 ceph-mon[32799]: pgmap v316: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:41.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:41 smithi019 ceph-mon[29209]: pgmap v316: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:43.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:43 smithi169 ceph-mon[33049]: pgmap v317: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:43.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:43 smithi112 ceph-mon[32799]: pgmap v317: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:43.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:43 smithi019 ceph-mon[29209]: pgmap v317: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:43.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T11:04:43.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T11:04:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.780+0000 7efe042d8640 1 -- 172.21.15.19:0/2858288278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc1069b0 msgr2=0x7efdfc106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.780+0000 7efe042d8640 1 --2- 172.21.15.19:0/2858288278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc1069b0 0x7efdfc106de0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec02a410 comp rx=0 tx=0).stop 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 -- 172.21.15.19:0/2858288278 shutdown_connections 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 --2- 172.21.15.19:0/2858288278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdfc107470 0x7efdfc10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 --2- 172.21.15.19:0/2858288278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc1069b0 0x7efdfc106de0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 --2- 172.21.15.19:0/2858288278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102420 0x7efdfc102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 -- 172.21.15.19:0/2858288278 >> 172.21.15.19:0/2858288278 conn(0x7efdfc077920 msgr2=0x7efdfc077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 -- 172.21.15.19:0/2858288278 shutdown_connections 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 -- 172.21.15.19:0/2858288278 wait complete. 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 Processor -- start 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.781+0000 7efe042d8640 1 -- start start 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe042d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102420 0x7efdfc19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe042d8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdfc1069b0 0x7efdfc19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe042d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc107470 0x7efdfc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe042d8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc111b00 con 0x7efdfc107470 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe042d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc111980 con 0x7efdfc102420 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe042d8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc111c80 con 0x7efdfc1069b0 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc107470 0x7efdfc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc107470 0x7efdfc19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52974/0 (socket says 172.21.15.19:52974) 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 -- 172.21.15.19:0/3028488389 learned_addr learned my addr 172.21.15.19:0/3028488389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02074640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102420 0x7efdfc19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe01873640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdfc1069b0 0x7efdfc19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 -- 172.21.15.19:0/3028488389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdfc1069b0 msgr2=0x7efdfc19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdfc1069b0 0x7efdfc19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 -- 172.21.15.19:0/3028488389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102420 msgr2=0x7efdfc19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102420 0x7efdfc19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 -- 172.21.15.19:0/3028488389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc19f5f0 con 0x7efdfc107470 2024-04-25T11:04:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe01873640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdfc1069b0 0x7efdfc19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.782+0000 7efe02875640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc107470 0x7efdfc19efb0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efdf400eb10 tx=0x7efdf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.783+0000 7efdf37fe640 1 -- 172.21.15.19:0/3028488389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4004b20 con 0x7efdfc107470 2024-04-25T11:04:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.783+0000 7efdf37fe640 1 -- 172.21.15.19:0/3028488389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf4004cc0 con 0x7efdfc107470 2024-04-25T11:04:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.783+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc19f8e0 con 0x7efdfc107470 2024-04-25T11:04:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.783+0000 7efdf37fe640 1 -- 172.21.15.19:0/3028488389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4010670 con 0x7efdfc107470 2024-04-25T11:04:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.783+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1a7260 con 0x7efdfc107470 2024-04-25T11:04:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.785+0000 7efdf37fe640 1 -- 172.21.15.19:0/3028488389 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7efdf4005020 con 0x7efdfc107470 2024-04-25T11:04:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 Processor -- start 2024-04-25T11:04:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 -- start start 2024-04-25T11:04:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c6793a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c6793fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efdf37fe640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7efde80afc60 0x7efde80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8c66f9d80 con 0x55a8c63d26b0 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a8c66b38b0 con 0x55a8c63ca4d0 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe0b188740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a8c66f5d40 con 0x55a8c63f3390 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe01873640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7efde80afc60 0x7efde80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52984/0 (socket says 172.21.15.19:52984) 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe02875640 1 -- 172.21.15.19:0/1393769172 learned_addr learned my addr 172.21.15.19:0/1393769172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe02074640 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c6793fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efe01873640 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c6793a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.786+0000 7efdf37fe640 1 -- 172.21.15.19:0/3028488389 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7efdf40df620 con 0x7efdfc107470 2024-04-25T11:04:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe02074640 1 -- 172.21.15.19:0/1393769172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 msgr2=0x55a8c6793a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe02074640 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c6793a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe02074640 1 -- 172.21.15.19:0/1393769172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 msgr2=0x55a8c67a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe02074640 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe02074640 1 -- 172.21.15.19:0/1393769172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c6684a40 con 0x55a8c63ca4d0 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe02875640 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe01873640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7efde80afc60 0x7efde80b2170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efdec005f70 tx=0x7efdec000bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efe02074640 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c6793fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efdf800ac90 tx=0x7efdf8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fefcf477263ba8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efdf0bfe640 1 -- 172.21.15.19:0/1393769172 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8004250 con 0x55a8c63ca4d0 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efdf0bfe640 1 -- 172.21.15.19:0/1393769172 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf800ae40 con 0x55a8c63ca4d0 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.787+0000 7efdf0bfe640 1 -- 172.21.15.19:0/1393769172 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8018690 con 0x55a8c63ca4d0 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.788+0000 7efe0b188740 1 -- 172.21.15.19:0/1393769172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 msgr2=0x55a8c6793fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.788+0000 7efe0b188740 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c6793fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efdf800ac90 tx=0x7efdf8005f90 comp rx=0 tx=0).stop 2024-04-25T11:04:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.788+0000 7efe0b188740 1 -- 172.21.15.19:0/1393769172 shutdown_connections 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.788+0000 7efe0b188740 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.788+0000 7efe0b188740 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c6793fa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.788+0000 7efe0b188740 1 --2- 172.21.15.19:0/1393769172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c6793a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.788+0000 7efe0b188740 1 -- 172.21.15.19:0/1393769172 >> 172.21.15.19:0/1393769172 conn(0x55a8c63e74e0 msgr2=0x55a8c6790210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 -- 172.21.15.19:0/1393769172 shutdown_connections 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 -- 172.21.15.19:0/1393769172 wait complete. 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 -- start start 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 Processor -- start 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63ca4d0 0x55a8c679e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63d26b0 0x55a8c679eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8c67a0fe0 con 0x55a8c63f3390 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a8c6680d10 con 0x55a8c63d26b0 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a8c67a1160 con 0x55a8c63ca4d0 2024-04-25T11:04:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.789+0000 7efe0b188740 10 client.0 _ll_register_callbacks cb 0x55a8c66ad210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52986/0 (socket says 172.21.15.19:52986) 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 -- 172.21.15.19:0/3642325925 learned_addr learned my addr 172.21.15.19:0/3642325925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe01873640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63d26b0 0x55a8c679eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02074640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63ca4d0 0x55a8c679e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 -- 172.21.15.19:0/3642325925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63ca4d0 msgr2=0x55a8c679e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63ca4d0 0x55a8c679e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 -- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63d26b0 msgr2=0x55a8c679eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63d26b0 0x55a8c679eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 -- 172.21.15.19:0/3642325925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c64d57d0 con 0x55a8c63f3390 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe01873640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63d26b0 0x55a8c679eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe02875640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67a4430 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7efdf40049d0 tx=0x7efdf401c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeeea3594ec2393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efde5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efde5ffb640 1 -- 172.21.15.19:0/3642325925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf40a7530 con 0x55a8c63f3390 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efde5ffb640 1 -- 172.21.15.19:0/3642325925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7efdf40a3d50 con 0x55a8c63f3390 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8c6707990 con 0x55a8c63f3390 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efde5ffb640 1 -- 172.21.15.19:0/3642325925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efdf40ca0c0 con 0x55a8c63f3390 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe0b188740 10 client.17070 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a8c640b3f0 con 0x55a8c63f3390 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.790+0000 7efde5ffb640 1 -- 172.21.15.19:0/3642325925 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8c6707990 con 0x55a8c63f3390 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efde5ffb640 1 -- 172.21.15.19:0/3642325925 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7efdf407e0d0 con 0x55a8c63f3390 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efe0b188740 10 client.17070 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efe0b188740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efe0b188740 20 client.17070 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efe0b188740 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63b1f60 0x55a8c67b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efe0b188740 4 client.17070 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c6701800 con 0x55a8c63b1f60 2024-04-25T11:04:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.791+0000 7efe02074640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63b1f60 0x55a8c67b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efde5ffb640 10 client.17070 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe02074640 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63b1f60 0x55a8c67b0710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb5d7462dbd2549 server_cookie=1a55790659a3ac0b in_seq=0 out_seq=0 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efde5ffb640 1 -- 172.21.15.19:0/3642325925 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8c6701800 con 0x55a8c63b1f60 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efde5ffb640 10 client.17070 handle_command_reply: tid=0 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe0b188740 1 client.17070 shutdown 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 msgr2=0x55a8c67a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe0b188740 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67a4430 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7efdf40049d0 tx=0x7efdf401c7b0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 shutdown_connections 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe0b188740 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63b1f60 0x55a8c67b0710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe0b188740 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67a4430 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efe0b188740 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63d26b0 0x55a8c679eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.792+0000 7efde5ffb640 0 client.17070 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.793+0000 7efe0b188740 1 --2- 172.21.15.19:0/3642325925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63ca4d0 0x55a8c679e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.793+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 >> 172.21.15.19:0/3642325925 conn(0x55a8c63e74e0 msgr2=0x55a8c6795780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.793+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 shutdown_connections 2024-04-25T11:04:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.793+0000 7efe0b188740 1 -- 172.21.15.19:0/3642325925 wait complete. 2024-04-25T11:04:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.793+0000 7efe0b188740 20 client.17070 trim_cache size 0 max 16384 2024-04-25T11:04:43.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 Processor -- start 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 -- start start 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63d26b0 0x55a8c679db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1adb70 con 0x55a8c63f3390 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1b03d0 con 0x55a8c63ca4d0 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe0b188740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1ae1c0 con 0x55a8c63d26b0 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52996/0 (socket says 172.21.15.19:52996) 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 -- 172.21.15.19:0/1642540662 learned_addr learned my addr 172.21.15.19:0/1642540662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:43.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 -- 172.21.15.19:0/1642540662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63d26b0 msgr2=0x55a8c679db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe01873640 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63d26b0 0x55a8c679db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02074640 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63d26b0 0x55a8c679db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 -- 172.21.15.19:0/1642540662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 msgr2=0x55a8c67a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02875640 1 -- 172.21.15.19:0/1642540662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c63cb0f0 con 0x55a8c63f3390 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe01873640 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63d26b0 0x55a8c679db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.811+0000 7efe02074640 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efe02875640 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67b5ed0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7efdf40a76f0 tx=0x7efdf400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849eb4f40c7b7e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efde7fff640 1 -- 172.21.15.19:0/1642540662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4013190 con 0x55a8c63f3390 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efde7fff640 1 -- 172.21.15.19:0/1642540662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf40a7270 con 0x55a8c63f3390 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efde7fff640 1 -- 172.21.15.19:0/1642540662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf40ad2a0 con 0x55a8c63f3390 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efe0b188740 1 -- 172.21.15.19:0/1642540662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 msgr2=0x55a8c67b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efe0b188740 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67b5ed0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7efdf40a76f0 tx=0x7efdf400efd0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efe0b188740 1 -- 172.21.15.19:0/1642540662 shutdown_connections 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efe0b188740 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63f3390 0x55a8c67b5ed0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efe0b188740 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63d26b0 0x55a8c679db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.812+0000 7efe0b188740 1 --2- 172.21.15.19:0/1642540662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 -- 172.21.15.19:0/1642540662 >> 172.21.15.19:0/1642540662 conn(0x55a8c63e74e0 msgr2=0x55a8c6795cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 -- 172.21.15.19:0/1642540662 shutdown_connections 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 -- 172.21.15.19:0/1642540662 wait complete. 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 -- start start 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 Processor -- start 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c67a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1ae890 con 0x55a8c63d26b0 2024-04-25T11:04:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1afd00 con 0x55a8c63ca4d0 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1aef60 con 0x55a8c63f3390 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe0b188740 10 client.0 _ll_register_callbacks cb 0x7efdf40dccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe01873640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe01873640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53002/0 (socket says 172.21.15.19:53002) 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.813+0000 7efe01873640 1 -- 172.21.15.19:0/3425926272 learned_addr learned my addr 172.21.15.19:0/3425926272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe01873640 1 -- 172.21.15.19:0/3425926272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 msgr2=0x55a8c67a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe02875640 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c67a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe02074640 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe01873640 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c67a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe01873640 1 -- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 msgr2=0x55a8c67aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe01873640 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67aaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe01873640 1 -- 172.21.15.19:0/3425926272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c66ba4b0 con 0x55a8c63d26b0 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe01873640 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67ab010 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceec7f04d55fab30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efde5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efde5ffb640 1 -- 172.21.15.19:0/3425926272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec035040 con 0x55a8c63d26b0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efde5ffb640 1 -- 172.21.15.19:0/3425926272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7efdec031810 con 0x55a8c63d26b0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf40a7270 con 0x55a8c63d26b0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efde5ffb640 1 -- 172.21.15.19:0/3425926272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efdec059050 con 0x55a8c63d26b0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe0b188740 10 client.17076 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a8c68181d0 con 0x55a8c63d26b0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.814+0000 7efde5ffb640 1 -- 172.21.15.19:0/3425926272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf40a7270 con 0x55a8c63d26b0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efde5ffb640 1 -- 172.21.15.19:0/3425926272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7efdec049070 con 0x55a8c63d26b0 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe0b188740 10 client.17076 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe0b188740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe0b188740 20 client.17076 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe0b188740 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63873a0 0x55a8c681a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe0b188740 4 client.17076 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8c667c4a0 con 0x55a8c63873a0 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe02074640 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63873a0 0x55a8c681a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efde5ffb640 10 client.17076 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe02074640 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63873a0 0x55a8c681a9b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c9470d0f8a55772 server_cookie=37ab2f7530c594de in_seq=0 out_seq=0 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efde5ffb640 1 -- 172.21.15.19:0/3425926272 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a8c667c4a0 con 0x55a8c63873a0 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efde5ffb640 10 client.17076 handle_command_reply: tid=0 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.815+0000 7efe0b188740 1 client.17076 shutdown 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 msgr2=0x55a8c67ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67ab010 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec044000 comp rx=0 tx=0).stop 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 shutdown_connections 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x55a8c63873a0 0x55a8c681a9b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8c63f3390 0x55a8c67a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c63d26b0 0x55a8c67ab010 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efde5ffb640 0 client.17076 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 --2- 172.21.15.19:0/3425926272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8c63ca4d0 0x55a8c67aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 >> 172.21.15.19:0/3425926272 conn(0x55a8c63e74e0 msgr2=0x55a8c6795cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 shutdown_connections 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 1 -- 172.21.15.19:0/3425926272 wait complete. 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe0b188740 20 client.17076 trim_cache size 0 max 16384 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:04:43.824 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:43.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7efde80afc60 msgr2=0x7efde80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7efde80afc60 0x7efde80b2170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efdec005f70 tx=0x7efdec000bc0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc107470 msgr2=0x7efdfc19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc107470 0x7efdfc19efb0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efdf400eb10 tx=0x7efdf400c2d0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 shutdown_connections 2024-04-25T11:04:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7efde80afc60 0x7efde80b2170 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc107470 0x7efdfc19efb0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdfc1069b0 0x7efdfc19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 --2- 172.21.15.19:0/3028488389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102420 0x7efdfc19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 >> 172.21.15.19:0/3028488389 conn(0x7efdfc077920 msgr2=0x7efdfc1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 shutdown_connections 2024-04-25T11:04:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.816+0000 7efe042d8640 1 -- 172.21.15.19:0/3028488389 wait complete. 2024-04-25T11:04:43.827 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:04:43.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:04:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 -- 172.21.15.19:0/578049182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 msgr2=0x7fb0f0102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 --2- 172.21.15.19:0/578049182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f0102f70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009900 tx=0x7fb0e402a410 comp rx=0 tx=0).stop 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 -- 172.21.15.19:0/578049182 shutdown_connections 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 --2- 172.21.15.19:0/578049182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f010d3a0 0x7fb0f010f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 --2- 172.21.15.19:0/578049182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f0102f70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 --2- 172.21.15.19:0/578049182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f0108a40 0x7fb0f0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 -- 172.21.15.19:0/578049182 >> 172.21.15.19:0/578049182 conn(0x7fb0f0078040 msgr2=0x7fb0f0078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 -- 172.21.15.19:0/578049182 shutdown_connections 2024-04-25T11:04:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.940+0000 7fb0f48af640 1 -- 172.21.15.19:0/578049182 wait complete. 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 Processor -- start 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 -- start start 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f0108a40 0x7fb0f019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f010d3a0 0x7fb0f019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0112c00 con 0x7fb0f0102b60 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0112a80 con 0x7fb0f0108a40 2024-04-25T11:04:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0f48af640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0112d80 con 0x7fb0f010d3a0 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53014/0 (socket says 172.21.15.19:53014) 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 -- 172.21.15.19:0/1839384457 learned_addr learned my addr 172.21.15.19:0/1839384457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0edd9b640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f0108a40 0x7fb0f019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0eed9d640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f010d3a0 0x7fb0f019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 -- 172.21.15.19:0/1839384457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f010d3a0 msgr2=0x7fb0f019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f010d3a0 0x7fb0f019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 -- 172.21.15.19:0/1839384457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f0108a40 msgr2=0x7fb0f019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f0108a40 0x7fb0f019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.941+0000 7fb0ee59c640 1 -- 172.21.15.19:0/1839384457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f019f510 con 0x7fb0f0102b60 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.942+0000 7fb0edd9b640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f0108a40 0x7fb0f019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.942+0000 7fb0ee59c640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f019a6b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e9c0 tx=0x7fb0e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.942+0000 7fb0df7fe640 1 -- 172.21.15.19:0/1839384457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000cd80 con 0x7fb0f0102b60 2024-04-25T11:04:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.942+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f019f800 con 0x7fb0f0102b60 2024-04-25T11:04:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.942+0000 7fb0df7fe640 1 -- 172.21.15.19:0/1839384457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e00054d0 con 0x7fb0f0102b60 2024-04-25T11:04:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.942+0000 7fb0df7fe640 1 -- 172.21.15.19:0/1839384457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0010430 con 0x7fb0f0102b60 2024-04-25T11:04:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.942+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f01a7180 con 0x7fb0f0102b60 2024-04-25T11:04:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.943+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f006a190 con 0x7fb0f0102b60 2024-04-25T11:04:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.945+0000 7fb0df7fe640 1 -- 172.21.15.19:0/1839384457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb0e0004fb0 con 0x7fb0f0102b60 2024-04-25T11:04:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.945+0000 7fb0df7fe640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb0d80afc60 0x7fb0d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.945+0000 7fb0df7fe640 1 -- 172.21.15.19:0/1839384457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fb0e00df760 con 0x7fb0f0102b60 2024-04-25T11:04:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.945+0000 7fb0edd9b640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb0d80afc60 0x7fb0d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.946+0000 7fb0edd9b640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb0d80afc60 0x7fb0d80b2170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009230 tx=0x7fb0e4000bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:43.949+0000 7fb0df7fe640 1 -- 172.21.15.19:0/1839384457 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0e0053ff0 con 0x7fb0f0102b60 2024-04-25T11:04:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.194+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0f019ba00 con 0x7fb0f0102b60 2024-04-25T11:04:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.198+0000 7fb0df7fe640 1 -- 172.21.15.19:0/1839384457 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7fb0e00a8000 con 0x7fb0f0102b60 2024-04-25T11:04:44.198 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:04:44.199 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:04:44.199 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:04:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb0d80afc60 msgr2=0x7fb0d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb0d80afc60 0x7fb0d80b2170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009230 tx=0x7fb0e4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 msgr2=0x7fb0f019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f019a6b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e9c0 tx=0x7fb0e000ee80 comp rx=0 tx=0).stop 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 shutdown_connections 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fb0d80afc60 0x7fb0d80b2170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f010d3a0 0x7fb0f019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f0108a40 0x7fb0f019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 --2- 172.21.15.19:0/1839384457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f0102b60 0x7fb0f019a6b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 >> 172.21.15.19:0/1839384457 conn(0x7fb0f0078040 msgr2=0x7fb0f0074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 shutdown_connections 2024-04-25T11:04:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.200+0000 7fb0f48af640 1 -- 172.21.15.19:0/1839384457 wait complete. 2024-04-25T11:04:44.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T11:04:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 -- 172.21.15.19:0/701257459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10def0 msgr2=0x7f933c1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 --2- 172.21.15.19:0/701257459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10def0 0x7f933c1103e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9334009ca0 tx=0x7f933402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 -- 172.21.15.19:0/701257459 shutdown_connections 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 --2- 172.21.15.19:0/701257459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10def0 0x7f933c1103e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 --2- 172.21.15.19:0/701257459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f933c10b500 0x7f933c10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 --2- 172.21.15.19:0/701257459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c104aa0 0x7f933c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 -- 172.21.15.19:0/701257459 >> 172.21.15.19:0/701257459 conn(0x7f933c078000 msgr2=0x7f933c078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 -- 172.21.15.19:0/701257459 shutdown_connections 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 -- 172.21.15.19:0/701257459 wait complete. 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.328+0000 7f9340f9d640 1 Processor -- start 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9340f9d640 1 -- start start 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9340f9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f933c104aa0 0x7f933c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9340f9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10b500 0x7f933c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9340f9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c10def0 0x7f933c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9340f9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f933c113b50 con 0x7f933c10b500 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9340f9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1139d0 con 0x7f933c104aa0 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9340f9d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f933c113cd0 con 0x7f933c10def0 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10b500 0x7f933c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10b500 0x7f933c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53040/0 (socket says 172.21.15.19:53040) 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 -- 172.21.15.19:0/3636360008 learned_addr learned my addr 172.21.15.19:0/3636360008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f933b7fe640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f933c104aa0 0x7f933c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f933bfff640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c10def0 0x7f933c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 -- 172.21.15.19:0/3636360008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c10def0 msgr2=0x7f933c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c10def0 0x7f933c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 -- 172.21.15.19:0/3636360008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f933c104aa0 msgr2=0x7f933c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f933c104aa0 0x7f933c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f9333fff640 1 -- 172.21.15.19:0/3636360008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c19f390 con 0x7f933c10b500 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.329+0000 7f933b7fe640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f933c104aa0 0x7f933c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.330+0000 7f9333fff640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10b500 0x7f933c19aa90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f932400e9f0 tx=0x7f932400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.330+0000 7f93397fa640 1 -- 172.21.15.19:0/3636360008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932400cd80 con 0x7f933c10b500 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.330+0000 7f93397fa640 1 -- 172.21.15.19:0/3636360008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93240108e0 con 0x7f933c10b500 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.330+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c19f680 con 0x7f933c10b500 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.330+0000 7f93397fa640 1 -- 172.21.15.19:0/3636360008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324018980 con 0x7f933c10b500 2024-04-25T11:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.330+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c1a7000 con 0x7f933c10b500 2024-04-25T11:04:44.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.332+0000 7f93397fa640 1 -- 172.21.15.19:0/3636360008 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9324010430 con 0x7f933c10b500 2024-04-25T11:04:44.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.333+0000 7f93397fa640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f93280afcd0 0x7f93280b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.333+0000 7f933b7fe640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f93280afcd0 0x7f93280b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.333+0000 7f93397fa640 1 -- 172.21.15.19:0/3636360008 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f93240df310 con 0x7f933c10b500 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.333+0000 7f934918f740 1 Processor -- start 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.333+0000 7f934918f740 1 -- start start 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933b7fe640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f93280afcd0 0x7f93280b21e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f932c009fc0 tx=0x7f932c005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f934918f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3ff6390 0x555cb43979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f934918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb4397f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f934918f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fd56b0 0x555cb43a4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f934918f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555cb42fcd80 con 0x555cb3fd56b0 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f934918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555cb42b6a20 con 0x555cb3fcd4d0 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f934918f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555cb42f8d40 con 0x555cb3ff6390 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fd56b0 0x555cb43a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fd56b0 0x555cb43a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53052/0 (socket says 172.21.15.19:53052) 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3ff6390 0x555cb43979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 -- 172.21.15.19:0/2429401360 learned_addr learned my addr 172.21.15.19:0/2429401360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f9333fff640 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb4397f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 -- 172.21.15.19:0/2429401360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3ff6390 msgr2=0x555cb43979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3ff6390 0x555cb43979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 -- 172.21.15.19:0/2429401360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 msgr2=0x555cb4397f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb4397f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933bfff640 1 -- 172.21.15.19:0/2429401360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb42c1c30 con 0x555cb3fd56b0 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.334+0000 7f933b7fe640 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3ff6390 0x555cb43979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.335+0000 7f933bfff640 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fd56b0 0x555cb43a4870 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f933402f410 tx=0x7f933402f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3361d93deead07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.335+0000 7f9331ffb640 1 -- 172.21.15.19:0/2429401360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334031cf0 con 0x555cb3fd56b0 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.335+0000 7f9331ffb640 1 -- 172.21.15.19:0/2429401360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9334036030 con 0x555cb3fd56b0 2024-04-25T11:04:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.335+0000 7f9331ffb640 1 -- 172.21.15.19:0/2429401360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340042a0 con 0x555cb3fd56b0 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 -- 172.21.15.19:0/2429401360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fd56b0 msgr2=0x555cb43a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fd56b0 0x555cb43a4870 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f933402f410 tx=0x7f933402f440 comp rx=0 tx=0).stop 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 -- 172.21.15.19:0/2429401360 shutdown_connections 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fd56b0 0x555cb43a4870 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb4397f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 --2- 172.21.15.19:0/2429401360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3ff6390 0x555cb43979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 -- 172.21.15.19:0/2429401360 >> 172.21.15.19:0/2429401360 conn(0x555cb3fea4e0 msgr2=0x555cb4394190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 -- 172.21.15.19:0/2429401360 shutdown_connections 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 -- 172.21.15.19:0/2429401360 wait complete. 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 -- start start 2024-04-25T11:04:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.336+0000 7f934918f740 1 Processor -- start 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f934918f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fcd4d0 0x555cb43a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f934918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fd56b0 0x555cb43a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f934918f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3ff6390 0x555cb43ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f934918f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555cb43a4db0 con 0x555cb3ff6390 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f934918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555cb4283dc0 con 0x555cb3fd56b0 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f934918f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555cb43a4f30 con 0x555cb3fcd4d0 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f934918f740 10 client.0 _ll_register_callbacks cb 0x555cb429f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3ff6390 0x555cb43ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3ff6390 0x555cb43ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53058/0 (socket says 172.21.15.19:53058) 2024-04-25T11:04:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 -- 172.21.15.19:0/1782883833 learned_addr learned my addr 172.21.15.19:0/1782883833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f9333fff640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fd56b0 0x555cb43a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933b7fe640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fcd4d0 0x555cb43a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 -- 172.21.15.19:0/1782883833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fcd4d0 msgr2=0x555cb43a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fcd4d0 0x555cb43a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 -- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fd56b0 msgr2=0x555cb43a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fd56b0 0x555cb43a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 -- 172.21.15.19:0/1782883833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb400e3f0 con 0x555cb3ff6390 2024-04-25T11:04:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933b7fe640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fcd4d0 0x555cb43a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f9333fff640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fd56b0 0x555cb43a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f933bfff640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3ff6390 0x555cb43ae230 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f933402f500 tx=0x7f93340099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eabcc202753cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.337+0000 7f931affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f931affd640 1 -- 172.21.15.19:0/1782883833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933404a040 con 0x555cb3ff6390 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f931affd640 1 -- 172.21.15.19:0/1782883833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f9334031650 con 0x555cb3ff6390 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9334036030 con 0x555cb3ff6390 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f931affd640 1 -- 172.21.15.19:0/1782883833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f933405e050 con 0x555cb3ff6390 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 10 client.17094 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555cb40d87d0 con 0x555cb3ff6390 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f931affd640 1 -- 172.21.15.19:0/1782883833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9334036030 con 0x555cb3ff6390 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f931affd640 1 -- 172.21.15.19:0/1782883833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f933404d030 con 0x555cb3ff6390 2024-04-25T11:04:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 10 client.17094 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 20 client.17094 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3fb4f60 0x555cb43b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 4 client.17094 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.338+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cb430a990 con 0x555cb3fb4f60 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.339+0000 7f933b7fe640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3fb4f60 0x555cb43b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.339+0000 7f931affd640 10 client.17094 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.339+0000 7f933b7fe640 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3fb4f60 0x555cb43b4450 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47728fda46f5eac server_cookie=7bc49ef2642dd0cc in_seq=0 out_seq=0 2024-04-25T11:04:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.340+0000 7f931affd640 1 -- 172.21.15.19:0/1782883833 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555cb430a990 con 0x555cb3fb4f60 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.340+0000 7f931affd640 10 client.17094 handle_command_reply: tid=0 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.340+0000 7f934918f740 1 client.17094 shutdown 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3ff6390 msgr2=0x555cb43ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3ff6390 0x555cb43ae230 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f933402f500 tx=0x7f93340099c0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 shutdown_connections 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3fb4f60 0x555cb43b4450 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3ff6390 0x555cb43ae230 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fd56b0 0x555cb43a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 --2- 172.21.15.19:0/1782883833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fcd4d0 0x555cb43a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f931affd640 0 client.17094 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 >> 172.21.15.19:0/1782883833 conn(0x555cb3fea4e0 msgr2=0x555cb4399700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 shutdown_connections 2024-04-25T11:04:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 1 -- 172.21.15.19:0/1782883833 wait complete. 2024-04-25T11:04:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.341+0000 7f934918f740 20 client.17094 trim_cache size 0 max 16384 2024-04-25T11:04:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.354+0000 7f934918f740 1 Processor -- start 2024-04-25T11:04:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f934918f740 1 -- start start 2024-04-25T11:04:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f934918f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43ade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f934918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb439d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f934918f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb439d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f934918f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1ad700 con 0x555cb3fb4f60 2024-04-25T11:04:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f934918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aff60 con 0x555cb3fcd4d0 2024-04-25T11:04:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f934918f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1add50 con 0x555cb3fd56b0 2024-04-25T11:04:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53070/0 (socket says 172.21.15.19:53070) 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 -- 172.21.15.19:0/1580979570 learned_addr learned my addr 172.21.15.19:0/1580979570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f9333fff640 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb439d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933bfff640 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb439d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 -- 172.21.15.19:0/1580979570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 msgr2=0x555cb439d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb439d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 -- 172.21.15.19:0/1580979570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 msgr2=0x555cb439d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb439d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 -- 172.21.15.19:0/1580979570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb43aed30 con 0x555cb3fb4f60 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.355+0000 7f933b7fe640 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43ade90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f932c009e80 tx=0x7f932c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16504bcd8a006a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.356+0000 7f93317fa640 1 -- 172.21.15.19:0/1580979570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c030070 con 0x555cb3fb4f60 2024-04-25T11:04:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.356+0000 7f93317fa640 1 -- 172.21.15.19:0/1580979570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c00fbd0 con 0x555cb3fb4f60 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.356+0000 7f93317fa640 1 -- 172.21.15.19:0/1580979570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c02a400 con 0x555cb3fb4f60 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 -- 172.21.15.19:0/1580979570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 msgr2=0x555cb43ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43ade90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f932c009e80 tx=0x7f932c014000 comp rx=0 tx=0).stop 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 -- 172.21.15.19:0/1580979570 shutdown_connections 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb439d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb439d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 --2- 172.21.15.19:0/1580979570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43ade90 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 -- 172.21.15.19:0/1580979570 >> 172.21.15.19:0/1580979570 conn(0x555cb3fea4e0 msgr2=0x555cb43a2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 -- 172.21.15.19:0/1580979570 shutdown_connections 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 -- 172.21.15.19:0/1580979570 wait complete. 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 -- start start 2024-04-25T11:04:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.357+0000 7f934918f740 1 Processor -- start 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f934918f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f934918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb43a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f934918f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb43a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f934918f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1ae420 con 0x555cb3fb4f60 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f934918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1af890 con 0x555cb3fcd4d0 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f934918f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aeaf0 con 0x555cb3fd56b0 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f934918f740 10 client.0 _ll_register_callbacks cb 0x7f93240a2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f9333fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb43a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43aa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53080/0 (socket says 172.21.15.19:53080) 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 -- 172.21.15.19:0/2376833024 learned_addr learned my addr 172.21.15.19:0/2376833024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933bfff640 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb43a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 -- 172.21.15.19:0/2376833024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 msgr2=0x555cb43a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb43a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 -- 172.21.15.19:0/2376833024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 msgr2=0x555cb43a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb43a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 -- 172.21.15.19:0/2376833024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb43a5d80 con 0x555cb3fb4f60 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f933b7fe640 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43aa660 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f932c005af0 tx=0x7f932c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e29430fd53ef337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f931affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.358+0000 7f931affd640 1 -- 172.21.15.19:0/2376833024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c039030 con 0x555cb3fb4f60 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f931affd640 1 -- 172.21.15.19:0/2376833024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f932c009050 con 0x555cb3fb4f60 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932c00fbd0 con 0x555cb3fb4f60 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f931affd640 1 -- 172.21.15.19:0/2376833024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f932c045050 con 0x555cb3fb4f60 2024-04-25T11:04:44.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 10 client.17100 fetch_fsmap learned FSMap version 20 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555cb441b4b0 con 0x555cb3fb4f60 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f931affd640 1 -- 172.21.15.19:0/2376833024 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c00fbd0 con 0x555cb3fb4f60 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f931affd640 1 -- 172.21.15.19:0/2376833024 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f932c01a040 con 0x555cb3fb4f60 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 10 client.17100 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 20 client.17100 populate_metadata read hostname 'smithi019' 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3ff6390 0x555cb441db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 4 client.17100 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 --> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cb4304800 con 0x555cb3ff6390 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.359+0000 7f9333fff640 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3ff6390 0x555cb441db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f931affd640 10 client.17100 ms_handle_connect on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f9333fff640 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3ff6390 0x555cb441db60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8593ca3bbd373c42 server_cookie=631f6b1fbae7c386 in_seq=0 out_seq=0 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f931affd640 1 -- 172.21.15.19:0/2376833024 <== mds.0 v2:172.21.15.112:6838/850740995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cb4304800 con 0x555cb3ff6390 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f931affd640 10 client.17100 handle_command_reply: tid=0 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 client.17100 shutdown 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 msgr2=0x555cb43aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43aa660 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f932c005af0 tx=0x7f932c005760 comp rx=0 tx=0).stop 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 shutdown_connections 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.112:6838/850740995,v1:172.21.15.112:6839/850740995] conn(0x555cb3ff6390 0x555cb441db60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cb3fd56b0 0x555cb43a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cb3fcd4d0 0x555cb43a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f931affd640 0 client.17100 ms_handle_reset on v2:172.21.15.112:6838/850740995 2024-04-25T11:04:44.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 --2- 172.21.15.19:0/2376833024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb3fb4f60 0x555cb43aa660 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 >> 172.21.15.19:0/2376833024 conn(0x555cb3fea4e0 msgr2=0x555cb43a2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:44.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 shutdown_connections 2024-04-25T11:04:44.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.360+0000 7f934918f740 1 -- 172.21.15.19:0/2376833024 wait complete. 2024-04-25T11:04:44.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f934918f740 20 client.17100 trim_cache size 0 max 16384 2024-04-25T11:04:44.373 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T11:04:44.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f93280afcd0 msgr2=0x7f93280b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f93280afcd0 0x7f93280b21e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f932c009fc0 tx=0x7f932c005f90 comp rx=0 tx=0).stop 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10b500 msgr2=0x7f933c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10b500 0x7f933c19aa90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f932400e9f0 tx=0x7f932400eeb0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 shutdown_connections 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f93280afcd0 0x7f93280b21e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c10def0 0x7f933c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c10b500 0x7f933c19aa90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 --2- 172.21.15.19:0/3636360008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f933c104aa0 0x7f933c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 >> 172.21.15.19:0/3636360008 conn(0x7f933c078000 msgr2=0x7f933c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 shutdown_connections 2024-04-25T11:04:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:44.361+0000 7f9340f9d640 1 -- 172.21.15.19:0/3636360008 wait complete. 2024-04-25T11:04:44.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:44 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:04:44.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:44 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:04:44.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:44 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:04:45.377 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-25T11:04:45.377 INFO:tasks.fwd_scrub:done joining 2024-04-25T11:04:45.377 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T11:04:45.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:04:45.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:45 smithi169 ceph-mon[33049]: pgmap v318: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:45.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:45 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1839384457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 -- 172.21.15.19:0/840623083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc10d380 msgr2=0x7f44bc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 --2- 172.21.15.19:0/840623083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc10d380 0x7f44bc10f7a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009ca0 tx=0x7f44b402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 -- 172.21.15.19:0/840623083 shutdown_connections 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 --2- 172.21.15.19:0/840623083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc10d380 0x7f44bc10f7a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 --2- 172.21.15.19:0/840623083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc102af0 0x7f44bc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 --2- 172.21.15.19:0/840623083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc108a20 0x7f44bc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 -- 172.21.15.19:0/840623083 >> 172.21.15.19:0/840623083 conn(0x7f44bc078110 msgr2=0x7f44bc078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 -- 172.21.15.19:0/840623083 shutdown_connections 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.516+0000 7f44c320c640 1 -- 172.21.15.19:0/840623083 wait complete. 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c320c640 1 Processor -- start 2024-04-25T11:04:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c320c640 1 -- start start 2024-04-25T11:04:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c320c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc102af0 0x7f44bc1a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c320c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc108a20 0x7f44bc1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc108a20 0x7f44bc1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c0fa8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc102af0 0x7f44bc1a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc108a20 0x7f44bc1a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53102/0 (socket says 172.21.15.19:53102) 2024-04-25T11:04:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44b3fff640 1 -- 172.21.15.19:0/82689759 learned_addr learned my addr 172.21.15.19:0/82689759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c320c640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc10d380 0x7f44bc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc112be0 con 0x7f44bc108a20 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.517+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc112a60 con 0x7f44bc10d380 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc112d60 con 0x7f44bc102af0 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44c17a9640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc10d380 0x7f44bc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b3fff640 1 -- 172.21.15.19:0/82689759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc102af0 msgr2=0x7f44bc1a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b3fff640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc102af0 0x7f44bc1a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b3fff640 1 -- 172.21.15.19:0/82689759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc10d380 msgr2=0x7f44bc19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b3fff640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc10d380 0x7f44bc19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b3fff640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc19cd50 con 0x7f44bc108a20 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b3fff640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc108a20 0x7f44bc1a1970 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00e9f0 tx=0x7f44ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b1ffb640 1 -- 172.21.15.19:0/82689759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00ccc0 con 0x7f44bc108a20 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc19d040 con 0x7f44bc108a20 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b1ffb640 1 -- 172.21.15.19:0/82689759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44ac0035f0 con 0x7f44bc108a20 2024-04-25T11:04:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.518+0000 7f44b1ffb640 1 -- 172.21.15.19:0/82689759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00ce60 con 0x7f44bc108a20 2024-04-25T11:04:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.519+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a9350 con 0x7f44bc108a20 2024-04-25T11:04:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.520+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc103cd0 con 0x7f44bc108a20 2024-04-25T11:04:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.521+0000 7f44b1ffb640 1 -- 172.21.15.19:0/82689759 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f44ac003aa0 con 0x7f44bc108a20 2024-04-25T11:04:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.522+0000 7f44b1ffb640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f44a40afc90 0x7f44a40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.522+0000 7f44b1ffb640 1 -- 172.21.15.19:0/82689759 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f44ac0df4c0 con 0x7f44bc108a20 2024-04-25T11:04:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.522+0000 7f44c0fa8640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f44a40afc90 0x7f44a40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.523+0000 7f44c0fa8640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f44a40afc90 0x7f44a40b21a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44bc19df00 tx=0x7f44b8005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.525+0000 7f44b1ffb640 1 -- 172.21.15.19:0/82689759 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44ac0a7d60 con 0x7f44bc108a20 2024-04-25T11:04:45.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:45 smithi112 ceph-mon[32799]: pgmap v318: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:45.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:45 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1839384457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:04:45.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:45 smithi019 ceph-mon[29209]: pgmap v318: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:45.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:45 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1839384457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:04:45.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.727+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44bc1a9580 con 0x7f44bc108a20 2024-04-25T11:04:45.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.728+0000 7f44b1ffb640 1 -- 172.21.15.19:0/82689759 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23125 (secure 0 0 0) 0x7f44ac0a7660 con 0x7f44bc108a20 2024-04-25T11:04:45.729 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:04:45.729 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T11:04:12.603369+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/48031724":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/2535542967":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/4264937365":"2024-04-26T10:54:29.901303+0000","172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3805408646":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6800/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f44a40afc90 msgr2=0x7f44a40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f44a40afc90 0x7f44a40b21a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44bc19df00 tx=0x7f44b8005f90 comp rx=0 tx=0).stop 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc108a20 msgr2=0x7f44bc1a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc108a20 0x7f44bc1a1970 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00e9f0 tx=0x7f44ac00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 shutdown_connections 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f44a40afc90 0x7f44a40b21a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc10d380 0x7f44bc19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44bc108a20 0x7f44bc1a1970 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 --2- 172.21.15.19:0/82689759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc102af0 0x7f44bc1a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.732+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 >> 172.21.15.19:0/82689759 conn(0x7f44bc078110 msgr2=0x7f44bc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.733+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 shutdown_connections 2024-04-25T11:04:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.733+0000 7f44c320c640 1 -- 172.21.15.19:0/82689759 wait complete. 2024-04-25T11:04:45.747 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:04:45.748 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:04:45.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.883+0000 7f917ac20640 1 -- 172.21.15.19:0/1922809661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 msgr2=0x7f9174102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.883+0000 7f917ac20640 1 --2- 172.21.15.19:0/1922809661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f9174102f70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f915c009900 tx=0x7f915c02a410 comp rx=0 tx=0).stop 2024-04-25T11:04:45.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 -- 172.21.15.19:0/1922809661 shutdown_connections 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 --2- 172.21.15.19:0/1922809661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f917410d3a0 0x7f917410f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 --2- 172.21.15.19:0/1922809661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f9174102f70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 --2- 172.21.15.19:0/1922809661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174108a40 0x7f9174102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 -- 172.21.15.19:0/1922809661 >> 172.21.15.19:0/1922809661 conn(0x7f91740fe260 msgr2=0x7f9174100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 -- 172.21.15.19:0/1922809661 shutdown_connections 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 -- 172.21.15.19:0/1922809661 wait complete. 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 Processor -- start 2024-04-25T11:04:45.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.884+0000 7f917ac20640 1 -- start start 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f917ac20640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f917419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f917ac20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174108a40 0x7f917419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f917ac20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f917410d3a0 0x7f917419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f917ac20640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9174112c00 con 0x7f9174102b60 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f917ac20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9174112a80 con 0x7f9174108a40 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f917ac20640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9174112d80 con 0x7f917410d3a0 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f917419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f917419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53126/0 (socket says 172.21.15.19:53126) 2024-04-25T11:04:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 -- 172.21.15.19:0/3953846194 learned_addr learned my addr 172.21.15.19:0/3953846194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f9173fff640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174108a40 0x7f917419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91791bd640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f917410d3a0 0x7f917419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 -- 172.21.15.19:0/3953846194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f917410d3a0 msgr2=0x7f917419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f917410d3a0 0x7f917419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 -- 172.21.15.19:0/3953846194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174108a40 msgr2=0x7f917419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174108a40 0x7f917419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 -- 172.21.15.19:0/3953846194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917419f5b0 con 0x7f9174102b60 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91791bd640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f917410d3a0 0x7f917419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f91789bc640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f917419a750 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f916400e9c0 tx=0x7f916400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f9171ffb640 1 -- 172.21.15.19:0/3953846194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916400cd80 con 0x7f9174102b60 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f9171ffb640 1 -- 172.21.15.19:0/3953846194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91640054d0 con 0x7f9174102b60 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917419f8a0 con 0x7f9174102b60 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.885+0000 7f9171ffb640 1 -- 172.21.15.19:0/3953846194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164010430 con 0x7f9174102b60 2024-04-25T11:04:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.886+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741a7220 con 0x7f9174102b60 2024-04-25T11:04:45.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.887+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f917406a160 con 0x7f9174102b60 2024-04-25T11:04:45.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.889+0000 7f9171ffb640 1 -- 172.21.15.19:0/3953846194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9164004fb0 con 0x7f9174102b60 2024-04-25T11:04:45.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.890+0000 7f9171ffb640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f91600afc60 0x7f91600b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.890+0000 7f9173fff640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f91600afc60 0x7f91600b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.890+0000 7f9171ffb640 1 -- 172.21.15.19:0/3953846194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f91640df770 con 0x7f9174102b60 2024-04-25T11:04:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.890+0000 7f9173fff640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f91600afc60 0x7f91600b2170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f915c009230 tx=0x7f915c031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:45.893+0000 7f9171ffb640 1 -- 172.21.15.19:0/3953846194 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9164053ff0 con 0x7f9174102b60 2024-04-25T11:04:46.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.083+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91741a73f0 con 0x7f9174102b60 2024-04-25T11:04:46.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.083+0000 7f9171ffb640 1 -- 172.21.15.19:0/3953846194 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f91640a7fb0 con 0x7f9174102b60 2024-04-25T11:04:46.084 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:04:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f91600afc60 msgr2=0x7f91600b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f91600afc60 0x7f91600b2170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f915c009230 tx=0x7f915c031400 comp rx=0 tx=0).stop 2024-04-25T11:04:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 msgr2=0x7f917419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f917419a750 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f916400e9c0 tx=0x7f916400ee80 comp rx=0 tx=0).stop 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 shutdown_connections 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f91600afc60 0x7f91600b2170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f917410d3a0 0x7f917419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174108a40 0x7f917419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 --2- 172.21.15.19:0/3953846194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9174102b60 0x7f917419a750 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 >> 172.21.15.19:0/3953846194 conn(0x7f91740fe260 msgr2=0x7f9174100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 shutdown_connections 2024-04-25T11:04:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.086+0000 7f917ac20640 1 -- 172.21.15.19:0/3953846194 wait complete. 2024-04-25T11:04:46.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:04:46.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 -- 172.21.15.19:0/3698636340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258102af0 msgr2=0x7f5258102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 --2- 172.21.15.19:0/3698636340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258102af0 0x7f5258102f00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5248009900 tx=0x7f524802a410 comp rx=0 tx=0).stop 2024-04-25T11:04:46.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 -- 172.21.15.19:0/3698636340 shutdown_connections 2024-04-25T11:04:46.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 --2- 172.21.15.19:0/3698636340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f525810d380 0x7f525810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 --2- 172.21.15.19:0/3698636340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258102af0 0x7f5258102f00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 --2- 172.21.15.19:0/3698636340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5258108a20 0x7f52581025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 -- 172.21.15.19:0/3698636340 >> 172.21.15.19:0/3698636340 conn(0x7f5258078100 msgr2=0x7f5258078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:46.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 -- 172.21.15.19:0/3698636340 shutdown_connections 2024-04-25T11:04:46.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.217+0000 7f52605f8640 1 -- 172.21.15.19:0/3698636340 wait complete. 2024-04-25T11:04:46.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 Processor -- start 2024-04-25T11:04:46.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 -- start start 2024-04-25T11:04:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5258102af0 0x7f52580ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258108a20 0x7f52580ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f525810d380 0x7f52580ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5258112be0 con 0x7f5258108a20 2024-04-25T11:04:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5258112a60 con 0x7f525810d380 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f52605f8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5258112d60 con 0x7f5258102af0 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f525db93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258108a20 0x7f52580ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f525db93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258108a20 0x7f52580ff790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53134/0 (socket says 172.21.15.19:53134) 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f525db93640 1 -- 172.21.15.19:0/693886079 learned_addr learned my addr 172.21.15.19:0/693886079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.218+0000 7f525e394640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5258102af0 0x7f52580ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525eb95640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f525810d380 0x7f52580ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525db93640 1 -- 172.21.15.19:0/693886079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5258102af0 msgr2=0x7f52580ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525db93640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5258102af0 0x7f52580ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525db93640 1 -- 172.21.15.19:0/693886079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f525810d380 msgr2=0x7f52580ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525db93640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f525810d380 0x7f52580ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525db93640 1 -- 172.21.15.19:0/693886079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52581a8a40 con 0x7f5258108a20 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525eb95640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f525810d380 0x7f52580ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:04:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f525db93640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258108a20 0x7f52580ff790 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f5248004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f524f7fe640 1 -- 172.21.15.19:0/693886079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524803f030 con 0x7f5258108a20 2024-04-25T11:04:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52581a8cd0 con 0x7f5258108a20 2024-04-25T11:04:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f524f7fe640 1 -- 172.21.15.19:0/693886079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5248003aa0 con 0x7f5258108a20 2024-04-25T11:04:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f524f7fe640 1 -- 172.21.15.19:0/693886079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248035070 con 0x7f5258108a20 2024-04-25T11:04:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.219+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52581a91c0 con 0x7f5258108a20 2024-04-25T11:04:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.220+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525806afa0 con 0x7f5258108a20 2024-04-25T11:04:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.222+0000 7f524f7fe640 1 -- 172.21.15.19:0/693886079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5248042030 con 0x7f5258108a20 2024-04-25T11:04:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.223+0000 7f524f7fe640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f52440afc60 0x7f52440b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.223+0000 7f524f7fe640 1 -- 172.21.15.19:0/693886079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f52480ff9a0 con 0x7f5258108a20 2024-04-25T11:04:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.223+0000 7f525e394640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f52440afc60 0x7f52440b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.223+0000 7f525e394640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f52440afc60 0x7f52440b2170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5258103a20 tx=0x7f5250005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:46.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.227+0000 7f524f7fe640 1 -- 172.21.15.19:0/693886079 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52480c81e0 con 0x7f5258108a20 2024-04-25T11:04:46.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.452+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52581a93c0 con 0x7f5258108a20 2024-04-25T11:04:46.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.453+0000 7f524f7fe640 1 -- 172.21.15.19:0/693886079 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f52480d0260 con 0x7f5258108a20 2024-04-25T11:04:46.453 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:04:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f52440afc60 msgr2=0x7f52440b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f52440afc60 0x7f52440b2170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5258103a20 tx=0x7f5250005f90 comp rx=0 tx=0).stop 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258108a20 msgr2=0x7f52580ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258108a20 0x7f52580ff790 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f5248004060 comp rx=0 tx=0).stop 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 shutdown_connections 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f52440afc60 0x7f52440b2170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f525810d380 0x7f52580ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5258108a20 0x7f52580ff790 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 --2- 172.21.15.19:0/693886079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5258102af0 0x7f52580ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 >> 172.21.15.19:0/693886079 conn(0x7f5258078100 msgr2=0x7f5258069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 shutdown_connections 2024-04-25T11:04:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.455+0000 7f52605f8640 1 -- 172.21.15.19:0/693886079 wait complete. 2024-04-25T11:04:46.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:04:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 -- 172.21.15.19:0/1900685128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 msgr2=0x7f56281025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 --2- 172.21.15.19:0/1900685128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f56281025b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5618009900 tx=0x7f561802a410 comp rx=0 tx=0).stop 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 -- 172.21.15.19:0/1900685128 shutdown_connections 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 --2- 172.21.15.19:0/1900685128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f562810d380 0x7f562810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 --2- 172.21.15.19:0/1900685128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5628102af0 0x7f5628102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 --2- 172.21.15.19:0/1900685128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f56281025b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 -- 172.21.15.19:0/1900685128 >> 172.21.15.19:0/1900685128 conn(0x7f5628078100 msgr2=0x7f5628078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 -- 172.21.15.19:0/1900685128 shutdown_connections 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.583+0000 7f562dee9640 1 -- 172.21.15.19:0/1900685128 wait complete. 2024-04-25T11:04:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 Processor -- start 2024-04-25T11:04:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 -- start start 2024-04-25T11:04:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5628102af0 0x7f5628199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f5628199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f562810d380 0x7f5628199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5628112be0 con 0x7f5628108a20 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5628112a60 con 0x7f5628102af0 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f562dee9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5628112d60 con 0x7f562810d380 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f5626ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f5628199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f5626ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f5628199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53140/0 (socket says 172.21.15.19:53140) 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f5626ffd640 1 -- 172.21.15.19:0/249193 learned_addr learned my addr 172.21.15.19:0/249193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f56277fe640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5628102af0 0x7f5628199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.584+0000 7f5627fff640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f562810d380 0x7f5628199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5626ffd640 1 -- 172.21.15.19:0/249193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f562810d380 msgr2=0x7f5628199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5626ffd640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f562810d380 0x7f5628199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5626ffd640 1 -- 172.21.15.19:0/249193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5628102af0 msgr2=0x7f5628199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5626ffd640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5628102af0 0x7f5628199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5626ffd640 1 -- 172.21.15.19:0/249193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56281a89c0 con 0x7f5628108a20 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5626ffd640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f5628199850 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f561c00e9f0 tx=0x7f561c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5624ff9640 1 -- 172.21.15.19:0/249193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c00ccc0 con 0x7f5628108a20 2024-04-25T11:04:46.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56281a8cb0 con 0x7f5628108a20 2024-04-25T11:04:46.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5624ff9640 1 -- 172.21.15.19:0/249193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f561c0035f0 con 0x7f5628108a20 2024-04-25T11:04:46.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f5624ff9640 1 -- 172.21.15.19:0/249193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c00ce60 con 0x7f5628108a20 2024-04-25T11:04:46.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.585+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56281a91a0 con 0x7f5628108a20 2024-04-25T11:04:46.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.586+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562806afa0 con 0x7f5628108a20 2024-04-25T11:04:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.588+0000 7f5624ff9640 1 -- 172.21.15.19:0/249193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f561c003aa0 con 0x7f5628108a20 2024-04-25T11:04:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.588+0000 7f5624ff9640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f56100afc60 0x7f56100b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.589+0000 7f56277fe640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f56100afc60 0x7f56100b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.589+0000 7f5624ff9640 1 -- 172.21.15.19:0/249193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f561c0df440 con 0x7f5628108a20 2024-04-25T11:04:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.589+0000 7f56277fe640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f56100afc60 0x7f56100b2170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5618000c00 tx=0x7f5618031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:46.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.592+0000 7f5624ff9640 1 -- 172.21.15.19:0/249193 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f561c0a7c80 con 0x7f5628108a20 2024-04-25T11:04:46.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.779+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f562802db50 con 0x7f5628108a20 2024-04-25T11:04:46.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.780+0000 7f5624ff9640 1 -- 172.21.15.19:0/249193 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f561c0a7580 con 0x7f5628108a20 2024-04-25T11:04:46.780 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:04:46.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f56100afc60 msgr2=0x7f56100b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f56100afc60 0x7f56100b2170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5618000c00 tx=0x7f5618031400 comp rx=0 tx=0).stop 2024-04-25T11:04:46.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 msgr2=0x7f5628199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f5628199850 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f561c00e9f0 tx=0x7f561c00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 shutdown_connections 2024-04-25T11:04:46.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f56100afc60 0x7f56100b2170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f562810d380 0x7f5628199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.781+0000 7f562dee9640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5628108a20 0x7f5628199850 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.782+0000 7f562dee9640 1 --2- 172.21.15.19:0/249193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5628102af0 0x7f5628199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.782+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 >> 172.21.15.19:0/249193 conn(0x7f5628078100 msgr2=0x7f56281040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:46.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.782+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 shutdown_connections 2024-04-25T11:04:46.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.782+0000 7f562dee9640 1 -- 172.21.15.19:0/249193 wait complete. 2024-04-25T11:04:46.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:46.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:46.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/82689759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:04:46.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:46.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:46.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:46.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3953846194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: pgmap v319: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: Unable to set osd_memory_target on smithi112 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/693886079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: Unable to set osd_memory_target on smithi019 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:04:46.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:46 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:46.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T11:04:46.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 -- 172.21.15.19:0/1909592505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410b680 msgr2=0x7fe49410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 --2- 172.21.15.19:0/1909592505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410b680 0x7fe49410daa0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe488009900 tx=0x7fe48802a480 comp rx=0 tx=0).stop 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 -- 172.21.15.19:0/1909592505 shutdown_connections 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 --2- 172.21.15.19:0/1909592505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe49410e070 0x7fe494110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 --2- 172.21.15.19:0/1909592505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410b680 0x7fe49410daa0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 --2- 172.21.15.19:0/1909592505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe494104c20 0x7fe494105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 -- 172.21.15.19:0/1909592505 >> 172.21.15.19:0/1909592505 conn(0x7fe494078040 msgr2=0x7fe494078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 -- 172.21.15.19:0/1909592505 shutdown_connections 2024-04-25T11:04:46.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.908+0000 7fe498a69640 1 -- 172.21.15.19:0/1909592505 wait complete. 2024-04-25T11:04:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 Processor -- start 2024-04-25T11:04:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 -- start start 2024-04-25T11:04:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe494104c20 0x7fe4941024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe49410b680 0x7fe494102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410e070 0x7fe494102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe494113cd0 con 0x7fe49410e070 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe494113b50 con 0x7fe49410b680 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe498a69640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe494113e50 con 0x7fe494104c20 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410e070 0x7fe494102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410e070 0x7fe494102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53172/0 (socket says 172.21.15.19:53172) 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe492d9d640 1 -- 172.21.15.19:0/3677583666 learned_addr learned my addr 172.21.15.19:0/3677583666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe491d9b640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe49410b680 0x7fe494102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe49259c640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe494104c20 0x7fe4941024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe492d9d640 1 -- 172.21.15.19:0/3677583666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe494104c20 msgr2=0x7fe4941024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe492d9d640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe494104c20 0x7fe4941024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe492d9d640 1 -- 172.21.15.19:0/3677583666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe49410b680 msgr2=0x7fe494102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.909+0000 7fe492d9d640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe49410b680 0x7fe494102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:46.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe492d9d640 1 -- 172.21.15.19:0/3677583666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4941a8a90 con 0x7fe49410e070 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe49259c640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe494104c20 0x7fe4941024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe492d9d640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410e070 0x7fe494102f60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009fd0 tx=0x7fe48c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe4837fe640 1 -- 172.21.15.19:0/3677583666 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c017070 con 0x7fe49410e070 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe4837fe640 1 -- 172.21.15.19:0/3677583666 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe48c003e20 con 0x7fe49410e070 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4941a8d80 con 0x7fe49410e070 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe4837fe640 1 -- 172.21.15.19:0/3677583666 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c010460 con 0x7fe49410e070 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.910+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4941a9190 con 0x7fe49410e070 2024-04-25T11:04:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.911+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49406a190 con 0x7fe49410e070 2024-04-25T11:04:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.912+0000 7fe4837fe640 1 -- 172.21.15.19:0/3677583666 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe48c003970 con 0x7fe49410e070 2024-04-25T11:04:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.913+0000 7fe4837fe640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe47c0afc60 0x7fe47c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:04:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.913+0000 7fe49259c640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe47c0afc60 0x7fe47c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:04:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.913+0000 7fe4837fe640 1 -- 172.21.15.19:0/3677583666 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fe48c0dedc0 con 0x7fe49410e070 2024-04-25T11:04:46.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.914+0000 7fe49259c640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe47c0afc60 0x7fe47c0b2170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe4840058c0 tx=0x7fe484005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:04:46.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:46.918+0000 7fe4837fe640 1 -- 172.21.15.19:0/3677583666 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe48c0a7600 con 0x7fe49410e070 2024-04-25T11:04:47.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/82689759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:04:47.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3953846194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: pgmap v319: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: Unable to set osd_memory_target on smithi112 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.002 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/693886079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: Unable to set osd_memory_target on smithi019 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:04:47.003 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:46 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/82689759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:04:47.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3953846194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: pgmap v319: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: Unable to set osd_memory_target on smithi112 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.060 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/693886079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: Unable to set osd_memory_target on smithi019 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:04:47.061 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:46 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' 2024-04-25T11:04:47.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.155+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe4941a94f0 con 0x7fe49410e070 2024-04-25T11:04:47.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.164+0000 7fe4837fe640 1 -- 172.21.15.19:0/3677583666 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+803659 (secure 0 0 0) 0x7fe48c0c9020 con 0x7fe49410e070 2024-04-25T11:04:47.175 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 26, 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 15210, 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2425110073 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-04-25T11:04:47.176 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2425110073 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/2425110073", 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-04-25T10:54:29.901635+0000", 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24907, 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-25T11:04:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:04:47.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T11:04:47.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.191 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:04:47.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:04:47.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.201 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T11:04:47.207 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:04:47.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:04:47.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T11:04:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T11:04:47.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T11:04:47.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:04:47.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T11:04:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:04:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:04:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T11:04:47.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T11:04:47.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T11:04:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T11:04:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T11:04:47.262 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:04:47.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:04:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:04:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.301 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.302 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:04:47.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:04:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:04:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:04:47.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:04:47.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:04:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:04:47.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.333 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T11:04:47.347 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T11:04:47.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.354 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T11:04:47.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:04:47.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.367 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T11:04:47.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.380 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:04:47.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:04:47.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.392 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.401 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.404 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:04:47.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.428 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.437 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.441 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:04:47.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.455 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.458 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.459 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.472 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.476 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.478 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:04:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.486 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T11:04:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T11:04:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.526 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.530 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.532 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T11:04:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T11:04:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T11:04:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T11:04:47.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T11:04:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T11:04:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.550 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.552 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.553 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.559 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.563 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.565 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.576 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T11:04:47.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T11:04:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T11:04:47.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.600 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.618 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.626 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24935, 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.634 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.642 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T11:04:47.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T11:04:47.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T11:04:47.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T11:04:47.647 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:04:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:04:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.656 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T11:04:47.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:04:47.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.667 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:04:47.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:04:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:04:47.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T11:04:47.674 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.692 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T11:04:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T11:04:47.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:04:47.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T11:04:47.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T11:04:47.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T11:04:47.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T11:04:47.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:04:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:04:47.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:04:47.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:04:47.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:04:47.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:04:47.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:04:47.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:04:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:04:47.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:04:47.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:04:47.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.790 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.805 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.807 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:04:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:04:47.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.831 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T11:04:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.834 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.835 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.837 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T11:04:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.849 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T11:04:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.853 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.854 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.856 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.858 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.866 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:04:47.875 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:04:47.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.884 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.888 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.892 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.899 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T11:04:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.906 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.907 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.907 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.908 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.908 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.908 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.908 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.909 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T11:04:47.915 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.923 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.925 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T11:04:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.936 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.940 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T11:04:47.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:04:47.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:47.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T11:04:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:04:47.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.957 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.963 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T11:04:47.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:04:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T11:04:47.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.983 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.986 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T11:04:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T11:04:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T11:04:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T11:04:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T11:04:48.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.022 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.023 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T11:04:48.026 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.030 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.033 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.034 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:04:48.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.045 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.046 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.047 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.050 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T11:04:48.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T11:04:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T11:04:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T11:04:48.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T11:04:48.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T11:04:48.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.064 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.066 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.075 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.079 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.088 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "restful", 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_schedule" 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:04:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.103 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.108 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T11:04:48.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T11:04:48.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T11:04:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.116 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T11:04:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T11:04:48.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.127 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.130 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.136 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.137 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T11:04:48.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:04:48.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T11:04:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:04:48.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.153 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.155 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T11:04:48.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T11:04:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T11:04:48.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T11:04:48.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T11:04:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:04:48.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:04:48.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T11:04:48.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:04:48.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:04:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:04:48.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T11:04:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T11:04:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.208 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:04:48.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:04:48.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:04:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:04:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.228 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.229 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:04:48.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:04:48.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.248 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.250 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.251 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:04:48.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:04:48.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T11:04:48.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.262 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.271 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T11:04:48.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T11:04:48.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.278 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T11:04:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T11:04:48.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:04:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.316 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.322 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:04:48.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T11:04:48.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.330 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T11:04:48.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.337 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T11:04:48.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T11:04:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.348 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T11:04:48.351 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.356 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.357 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.368 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T11:04:48.370 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.375 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.389 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T11:04:48.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:04:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:04:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T11:04:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T11:04:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T11:04:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.449 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.453 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.454 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T11:04:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T11:04:48.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T11:04:48.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.474 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.482 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.488 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T11:04:48.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.496 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T11:04:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T11:04:48.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T11:04:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T11:04:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:04:48.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:04:48.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.518 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T11:04:48.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.523 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.532 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T11:04:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:04:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T11:04:48.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T11:04:48.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T11:04:48.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T11:04:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T11:04:48.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.550 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T11:04:48.551 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T11:04:48.552 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T11:04:48.553 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T11:04:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 80, 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1868905359 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1929906083 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T11:04:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3884820588 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3409936330 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.557 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1279503079 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T11:04:48.558 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3415609301 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:04:48.559 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:48.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:47 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/249193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:04:48.560 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:47 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3677583666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:04:48.560 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:47 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/249193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:04:48.560 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:47 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3677583666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:04:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe47c0afc60 msgr2=0x7fe47c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe47c0afc60 0x7fe47c0b2170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe4840058c0 tx=0x7fe484005850 comp rx=0 tx=0).stop 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410e070 msgr2=0x7fe494102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410e070 0x7fe494102f60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009fd0 tx=0x7fe48c00b260 comp rx=0 tx=0).stop 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 shutdown_connections 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe47c0afc60 0x7fe47c0b2170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49410e070 0x7fe494102f60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe49410b680 0x7fe494102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 --2- 172.21.15.19:0/3677583666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe494104c20 0x7fe4941024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 >> 172.21.15.19:0/3677583666 conn(0x7fe494078040 msgr2=0x7fe494101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 shutdown_connections 2024-04-25T11:04:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:04:47.170+0000 7fe498a69640 1 -- 172.21.15.19:0/3677583666 wait complete. 2024-04-25T11:04:48.569 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T11:04:48.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T11:04:48.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:47 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/249193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:04:48.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:47 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3677583666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:04:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:04:48.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:48 smithi019 ceph-mon[29209]: pgmap v320: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:49.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:48 smithi169 ceph-mon[33049]: pgmap v320: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:49.058 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:48 smithi112 ceph-mon[32799]: pgmap v320: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2944, 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001891431, 2024-04-25T11:04:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000037570, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000622353, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000809488, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000269829 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 212, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 201, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1684560, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 123424, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.047923591, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.014000646, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.041345550, 2024-04-25T11:04:50.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004248959, 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 200, 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.011119914, 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000055599 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1684560, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 123424 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 153, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 183936, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T11:04:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.015338304, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.010695614, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002048035, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 150, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.011976419, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000079842 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:50.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 183936 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:04:50.701 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T11:04:50.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T11:04:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148994, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 31, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:04:50.704 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 214, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1654674, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 214, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 214, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1654674, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:50.706 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:50.707 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:50.707 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:50.707 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:50.707 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:51.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:51 smithi019 ceph-mon[29209]: pgmap v321: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:51.351 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T11:04:51.351 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T11:04:51.351 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T11:04:51.351 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T11:04:51.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T11:04:51.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:51 smithi169 ceph-mon[33049]: pgmap v321: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:51.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:51 smithi112 ceph-mon[32799]: pgmap v321: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:51.606 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:04:53.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:53 smithi019 ceph-mon[29209]: pgmap v322: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 218, 2024-04-25T11:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 207, 2024-04-25T11:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1686192, 2024-04-25T11:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 127264, 2024-04-25T11:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.051147695, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.014192200, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.043716542, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004452796, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 206, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.010704842, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000051965 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:53.266 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1686192, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 127264 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 6, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 160, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3264, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 185264, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.014945563, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.010229940, 2024-04-25T11:04:53.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002463065, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 158, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.013039270, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000082527 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 185264 2024-04-25T11:04:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.000393403, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000302757, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:53.269 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T11:04:53.270 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T11:04:53.271 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T11:04:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148977, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 30, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T11:04:53.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T11:04:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 220, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1655454, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 220, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 220, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1655454, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:53.276 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:53.276 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:53.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:53 smithi169 ceph-mon[33049]: pgmap v322: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:53.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:53 smithi112 ceph-mon[32799]: pgmap v322: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T11:04:53.863 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T11:04:53.863 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T11:04:53.863 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T11:04:53.863 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T11:04:53.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T11:04:54.114 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:04:55.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:55 smithi169 ceph-mon[33049]: pgmap v323: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:55.517 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:55 smithi019 ceph-mon[29209]: pgmap v323: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:55.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:55 smithi112 ceph-mon[32799]: pgmap v323: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:55.786 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001460975, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000318935, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-04-25T11:04:55.788 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 155, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 184288, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.014015785, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.009303429, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002225036, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 152, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.018141569, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000119352 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:55.789 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 184288 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 218, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 205, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1688048, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 125152, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T11:04:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.049914466, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.013958253, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.044059030, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004302201, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 205, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.010656880, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000051984 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1688048, 2024-04-25T11:04:55.791 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 125152 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:04:55.792 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:04:55.793 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:04:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 149034, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 32, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T11:04:55.795 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 216, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1654934, 2024-04-25T11:04:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 216, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 216, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1654934, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:55.797 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:56.496 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T11:04:56.496 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T11:04:56.496 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T11:04:56.496 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T11:04:56.496 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T11:04:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:04:57.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:57 smithi169 ceph-mon[33049]: pgmap v324: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:57.555 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:57 smithi019 ceph-mon[29209]: pgmap v324: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:57.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:57 smithi112 ceph-mon[32799]: pgmap v324: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T11:04:58.453 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:04:58.453 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2944, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001880500, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000048874, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000756420, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-04-25T11:04:58.454 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000911144, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000303714 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 160, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:04:58.455 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 185168, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.015262393, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.010636928, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001929934, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 157, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.014073673, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000089641 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:58.456 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 185168 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 220, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 209, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1686736, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 128544, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.048035457, 2024-04-25T11:04:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.014106647, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.041312522, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004523157, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 208, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.011121243, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000053467 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1686736, 2024-04-25T11:04:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 128544 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:04:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:04:58.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:04:58.461 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 149017, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 31, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:04:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 222, 2024-04-25T11:04:58.463 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1655714, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 222, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 222, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1655714, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:04:58.464 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:04:59.075 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T11:04:59.075 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T11:04:59.075 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T11:04:59.075 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T11:04:59.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T11:04:59.326 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:04:59.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:04:59 smithi019 ceph-mon[29209]: pgmap v325: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:59.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:04:59 smithi112 ceph-mon[32799]: pgmap v325: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:04:59.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:04:59 smithi169 ceph-mon[33049]: pgmap v325: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:05:01.141 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T11:05:01.141 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:01.141 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4139, 2024-04-25T11:05:01.141 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 3571, 2024-04-25T11:05:01.141 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1227749, 2024-04-25T11:05:01.141 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 9735190, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 18, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.396699630, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.133367238, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.099271227, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.129608195, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3569, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.552119408, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000154698 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:01.142 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 893, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000210417, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000235 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 318544, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 97408 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2947, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2797, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 903655, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 7514236, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 178, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 8, 2024-04-25T11:05:01.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.290748361, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.080872681, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.109729574, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.079823342, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2795, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.492113783, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000176069 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 374, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000061719, 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000165 2024-04-25T11:05:01.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 89392, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 132768 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 7292, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 11575, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3982213, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 6405393, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 10, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.616385420, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.221496726, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.363769045, 2024-04-25T11:05:01.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.101983555, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 11574, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.687789908, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000059425 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 13602, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002390633, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000175 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1778160, 2024-04-25T11:05:01.146 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 506144 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4533, 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2.380512002, 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000525151 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3582, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.243066952, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000067857 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 2597, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 2597, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2597, 2024-04-25T11:05:01.148 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2.925486080, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001126486 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 50, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 285, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 7, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 278, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-25T11:05:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 275, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 3, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 4, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 2, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 2659, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 2651, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-25T11:05:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 225, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 7, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 225, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 144, 2024-04-25T11:05:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 20, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 124, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 81, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 27, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 3638, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 2445, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 1213, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 13, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 1241 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.152 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 1, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 1212, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 1211, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:05:01.153 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 20, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-25T11:05:01.154 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 6191, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 3517, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 3517, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 2674, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 3446, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 16, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 12, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 12, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 4, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 1, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 4, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 12561515, 2024-04-25T11:05:01.155 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 18108477, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2507, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 4.196772086, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001674021 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 24, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 1255, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 1231, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 19, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 24, 2024-04-25T11:05:01.156 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 5, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 285, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 1246, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 961, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 4, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 1231, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 1227, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 133284, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 231824 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 2643, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 2597, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 32, 2024-04-25T11:05:01.157 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1210, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.287597748, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001064130 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 113, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.024071456, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000213021 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.158 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001110935, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000222187 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.159 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000406383, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000203191 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.160 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-25T11:05:01.161 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 11, 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001983884, 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000180353 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 12, 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.015063665, 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001255305 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 11, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.051019271, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004638115 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000340050, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000340050 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-25T11:05:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.007788847, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001298141 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002369914, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001184957 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-25T11:05:01.164 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.022511901, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003751983 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.017035598, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004258899 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.165 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001429376, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001429376 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.166 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.022989146, 2024-04-25T11:05:01.167 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003284163 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1206, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.469767906, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001218713 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:05:01.168 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 5, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 3, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 31, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 15, 2024-04-25T11:05:01.169 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 613, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:01.170 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:01.171 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 315381, 2024-04-25T11:05:01.172 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 134, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 616, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 7, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8032, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 167, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T11:05:01.173 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 302452, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2857, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 7302, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 14432607, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 37, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 7302, 2024-04-25T11:05:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7302, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 44.254828914, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.006060644 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7302, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 8001 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 7302, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 297, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 7005, 2024-04-25T11:05:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 29, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 289, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 52, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 2539, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 63, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 4013, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-25T11:05:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 245, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 578, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-25T11:05:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 88, 2024-04-25T11:05:01.178 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 12, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 170, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 0, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 23 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-25T11:05:01.179 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 34, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 13, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 3497, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 1211, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.180 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 14374, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 4753364, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 14374, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 14374, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 4753364, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.181 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7302, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 14554918, 2024-04-25T11:05:01.182 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7302, 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 14554918, 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T11:05:01.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7302, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 7302, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7302, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 7302, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 1211, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 122311, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 1211, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 32, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 122311, 2024-04-25T11:05:01.185 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55e9fa544750": { 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 84, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 6191, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 13914173, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:01.186 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 6191, 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2507, 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 13914089, 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:01.187 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T11:05:01.433 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:01 smithi112 ceph-mon[32799]: pgmap v326: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:05:01.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:01 smithi169 ceph-mon[33049]: pgmap v326: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:05:01.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:01 smithi019 ceph-mon[29209]: pgmap v326: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T11:05:01.945 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-04-25T11:05:01.945 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-25T11:05:01.945 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T11:05:01.946 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T11:05:01.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T11:05:02.190 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:05:03.292 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:03 smithi112 ceph-mon[32799]: pgmap v327: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:03.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:03 smithi169 ceph-mon[33049]: pgmap v327: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:03.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:03 smithi019 ceph-mon[29209]: pgmap v327: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:03.890 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.000923249, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000095795, 2024-04-25T11:05:03.891 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:03.892 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 159, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 184992, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.011208819, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.007554133, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001445174, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 156, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.020027185, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000128379 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:03.893 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 184992 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 225, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 212, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1690016, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 129152, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T11:05:03.894 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.049400554, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.013868470, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.041622911, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005585612, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 212, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012623917, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000059546 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1690016, 2024-04-25T11:05:03.895 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 129152 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:03.896 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:03.897 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148945, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 29, 2024-04-25T11:05:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T11:05:03.899 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 223, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1655908, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 223, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 223, 2024-04-25T11:05:03.900 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1655908, 2024-04-25T11:05:03.901 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:03.901 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:03.901 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:03.901 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:03.901 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:03.901 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:03.901 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T11:05:04.542 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T11:05:04.542 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T11:05:04.542 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T11:05:04.542 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T11:05:04.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T11:05:04.779 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:05:05.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:05 smithi169 ceph-mon[33049]: pgmap v328: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:05.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:05 smithi112 ceph-mon[32799]: pgmap v328: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:05.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:05 smithi019 ceph-mon[29209]: pgmap v328: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:06.364 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T11:05:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 2944, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001525341, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000033442, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000619775, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000776627, 2024-04-25T11:05:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000258875 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 219, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 208, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1686528, 2024-04-25T11:05:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 127424, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.050808814, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.014325558, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.043195818, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005346170, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 207, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.010094156, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000048764 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1686528, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 127424 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 159, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 184992, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.011052706, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.007228569, 2024-04-25T11:05:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001759104, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 156, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.017931756, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000114947 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 184992 2024-04-25T11:05:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148995, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 31, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T11:05:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 221, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1655648, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 221, 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 221, 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1655648, 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:06.375 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T11:05:07.041 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-25T11:05:07.041 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-25T11:05:07.042 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-25T11:05:07.042 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T11:05:07.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T11:05:07.283 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:05:07.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:07 smithi112 ceph-mon[32799]: pgmap v329: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:07.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:07 smithi169 ceph-mon[33049]: pgmap v329: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:07.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:07 smithi019 ceph-mon[29209]: pgmap v329: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:09.011 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T11:05:09.011 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001012971, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000227364, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:09.012 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2, 2024-04-25T11:05:09.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 162, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 185520, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.011817569, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.008036568, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001518440, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 159, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.022844297, 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000143674 2024-04-25T11:05:09.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 185520 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 228, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 215, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1690832, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 131072, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T11:05:09.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.051510987, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.014331812, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.044016551, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005535842, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 215, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.013545653, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000063003 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:09.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1690832, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 131072 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:09.017 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:09.018 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:09.019 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 149007, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T11:05:09.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 226, 2024-04-25T11:05:09.021 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1656298, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 226, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 226, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1656298, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T11:05:09.022 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T11:05:09.308 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:09 smithi112 ceph-mon[32799]: pgmap v330: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:09.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:09 smithi169 ceph-mon[33049]: pgmap v330: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:09.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:09 smithi019 ceph-mon[29209]: pgmap v330: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:09.677 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T11:05:09.677 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T11:05:09.677 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T11:05:09.677 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T11:05:09.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T11:05:09.934 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:05:11.475 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:11 smithi169 ceph-mon[33049]: pgmap v331: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:11.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:11 smithi112 ceph-mon[32799]: pgmap v331: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:11.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:11 smithi019 ceph-mon[29209]: pgmap v331: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T11:05:11.755 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T11:05:11.755 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:11.755 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 4, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 2944, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 448, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 0, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.001536270, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000043905, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.000669465, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000774028, 2024-04-25T11:05:11.756 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000258009 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 160, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:05:11.757 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 185168, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.011676519, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007706966, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.001690972, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 157, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.017800135, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000113376 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.758 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 185168 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 221, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 210, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1687072, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 128704, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 2, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-25T11:05:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.050289428, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.015140017, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.041070859, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005662272, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 209, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.011462892, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000054846 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.760 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1687072, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 128704 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 0, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:11.761 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:11.762 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:11.763 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 149001, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 31, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 928, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 2, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:11.765 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 223, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1655908, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 223, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 223, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1655908, 2024-04-25T11:05:11.766 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:11.767 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:11.767 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:11.767 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:11.767 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:11.767 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:11.767 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T11:05:12.417 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-25T11:05:12.417 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-25T11:05:12.417 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-25T11:05:12.418 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T11:05:12.418 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T11:05:12.673 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:05:13.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:13 smithi169 ceph-mon[33049]: pgmap v332: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:13.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:13 smithi112 ceph-mon[32799]: pgmap v332: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:13.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:13 smithi019 ceph-mon[29209]: pgmap v332: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T11:05:14.224 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T11:05:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 222, 2024-04-25T11:05:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 211, 2024-04-25T11:05:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1687344, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 129344, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 0, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.052006808, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.014397453, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.043372607, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005899889, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 210, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.011646870, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000055461 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1687344, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 129344 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 161, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 185344, 2024-04-25T11:05:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.011442241, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007102713, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.002185659, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 158, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.020882362, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000132166 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 185344 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 4, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 2944, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 448, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 2, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.000777331, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000037355, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.000732895, 2024-04-25T11:05:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000890052, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000296684 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T11:05:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 149024, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 31, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T11:05:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 928, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 2, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T11:05:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 224, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1656038, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 224, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 224, 2024-04-25T11:05:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1656038, 2024-04-25T11:05:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:14.235 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T11:05:14.903 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-25T11:05:14.903 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-25T11:05:14.903 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-25T11:05:14.903 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T11:05:14.904 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T11:05:15.145 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:05:15.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:15 smithi169 ceph-mon[33049]: pgmap v333: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T11:05:15.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:15 smithi112 ceph-mon[32799]: pgmap v333: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T11:05:15.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:15 smithi019 ceph-mon[29209]: pgmap v333: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1777, 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1765, 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 6352102, 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 620720, 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.230066801, 2024-04-25T11:05:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.064841827, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.094414197, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.068594834, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1764, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.073203677, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000041498 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1706848, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 131632 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 600, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 756, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 5684405, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 637150, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 7, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.075056488, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.024780314, 2024-04-25T11:05:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.021970909, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.020512587, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 754, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.045760296, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000060690 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448192 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 212, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 214, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 4246626, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 67423, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.022658116, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007126035, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.006659263, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.006959869, 2024-04-25T11:05:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 212, 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.009063608, 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000042752 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T11:05:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3888, 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.132891168, 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000034179 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-25T11:05:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 1236, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 345, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 891, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 743, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-04-25T11:05:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:05:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 7, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:05:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T11:05:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:05:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 6192, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-04-25T11:05:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 4, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 10, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 12561515, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 18108537, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 18108477, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 6192 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 23, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 1234, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 1211, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 18, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 23, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 5, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 1236, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 1242, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 6, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-25T11:05:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 123556, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215440 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T11:05:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T11:05:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 2, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 5, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 3, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 4, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 2852, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 189511, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 54, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T11:05:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8032, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 167, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 958828, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 5379, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 2344, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 26, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 4, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 2344, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2344, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.266971173, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000540516 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2344, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 2345 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 2344, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 2344, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 1498, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 843, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-04-25T11:05:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 2, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:05:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T11:05:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 88, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 11, 2024-04-25T11:05:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 5, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 5, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T11:05:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2585, 2024-04-25T11:05:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 16035308, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2585, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2585, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 16035308, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T11:05:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2344, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 13914243, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2344, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 13914243, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2344, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 2344, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2344, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2344, 2024-04-25T11:05:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T11:05:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55f4161bcc30": { 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T11:05:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:16.935 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T11:05:17.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:17 smithi169 ceph-mon[33049]: pgmap v334: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:05:17.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:17 smithi112 ceph-mon[32799]: pgmap v334: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:05:17.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:17 smithi019 ceph-mon[29209]: pgmap v334: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T11:05:17.667 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-04-25T11:05:17.667 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:05:17.667 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:05:17.667 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T11:05:17.667 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T11:05:17.906 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:05:19.333 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:19 smithi169 ceph-mon[33049]: pgmap v335: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:05:19.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:19 smithi112 ceph-mon[32799]: pgmap v335: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 227, 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 216, 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1688704, 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 132544, 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 0, 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.054393776, 2024-04-25T11:05:19.599 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.014436176, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.046541134, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005586912, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 215, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.012358336, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000057480 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1688704, 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 132544 2024-04-25T11:05:19.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 165, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 186048, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.011876677, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007990839, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.001544649, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 162, 2024-04-25T11:05:19.601 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.017634481, 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000108854 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 186048 2024-04-25T11:05:19.602 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 4, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 2944, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 448, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 2, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.000592083, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000029467, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.000526771, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3, 2024-04-25T11:05:19.603 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000719309, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000239769 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 0, 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T11:05:19.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:05:19.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T11:05:19.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 148955, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 30, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:05:19.607 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 928, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 2, 2024-04-25T11:05:19.608 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 229, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1656688, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 229, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T11:05:19.609 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 229, 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1656688, 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T11:05:19.610 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T11:05:19.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:19 smithi019 ceph-mon[29209]: pgmap v335: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:05:20.352 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T11:05:20.352 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T11:05:20.352 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T11:05:20.353 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T11:05:20.364 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T11:05:20.365 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:05:20.365 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:05:20.386 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:05:20.386 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:05:20.446 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:05:20.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.613+0000 7f8f42e25640 1 -- 172.21.15.19:0/3592558534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c104280 msgr2=0x7f8f3c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.613+0000 7f8f42e25640 1 --2- 172.21.15.19:0/3592558534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c104280 0x7f8f3c110650 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009ca0 tx=0x7f8f3402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.613+0000 7f8f42e25640 1 -- 172.21.15.19:0/3592558534 shutdown_connections 2024-04-25T11:05:20.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.613+0000 7f8f42e25640 1 --2- 172.21.15.19:0/3592558534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c104280 0x7f8f3c110650 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.613+0000 7f8f42e25640 1 --2- 172.21.15.19:0/3592558534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f3c109fd0 0x7f8f3c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.614+0000 7f8f42e25640 1 --2- 172.21.15.19:0/3592558534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c109b30 0x7f8f3c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.614+0000 7f8f42e25640 1 -- 172.21.15.19:0/3592558534 >> 172.21.15.19:0/3592558534 conn(0x7f8f3c0ff210 msgr2=0x7f8f3c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.614+0000 7f8f42e25640 1 -- 172.21.15.19:0/3592558534 shutdown_connections 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.614+0000 7f8f42e25640 1 -- 172.21.15.19:0/3592558534 wait complete. 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.614+0000 7f8f42e25640 1 Processor -- start 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.614+0000 7f8f42e25640 1 -- start start 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f42e25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f3c104280 0x7f8f3c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f42e25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c109b30 0x7f8f3c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f42e25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c109fd0 0x7f8f3c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f42e25640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c113d40 con 0x7f8f3c109b30 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f42e25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c113bc0 con 0x7f8f3c109fd0 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f42e25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c113ec0 con 0x7f8f3c104280 2024-04-25T11:05:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c109b30 0x7f8f3c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c109b30 0x7f8f3c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41186/0 (socket says 172.21.15.19:41186) 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 -- 172.21.15.19:0/2836332550 learned_addr learned my addr 172.21.15.19:0/2836332550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f40bc1640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f3c104280 0x7f8f3c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f413c2640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c109fd0 0x7f8f3c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 -- 172.21.15.19:0/2836332550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f3c104280 msgr2=0x7f8f3c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f3c104280 0x7f8f3c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 -- 172.21.15.19:0/2836332550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c109fd0 msgr2=0x7f8f3c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c109fd0 0x7f8f3c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 -- 172.21.15.19:0/2836332550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c19f560 con 0x7f8f3c109b30 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.615+0000 7f8f3bfff640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c109b30 0x7f8f3c19ac60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005d00 tx=0x7f8f2800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.616+0000 7f8f39ffb640 1 -- 172.21.15.19:0/2836332550 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28017070 con 0x7f8f3c109b30 2024-04-25T11:05:20.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.616+0000 7f8f39ffb640 1 -- 172.21.15.19:0/2836332550 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f28009c30 con 0x7f8f3c109b30 2024-04-25T11:05:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.616+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c19f850 con 0x7f8f3c109b30 2024-04-25T11:05:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.616+0000 7f8f39ffb640 1 -- 172.21.15.19:0/2836332550 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2800f620 con 0x7f8f3c109b30 2024-04-25T11:05:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.616+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c19fd50 con 0x7f8f3c109b30 2024-04-25T11:05:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.617+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c06a190 con 0x7f8f3c109b30 2024-04-25T11:05:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.618+0000 7f8f39ffb640 1 -- 172.21.15.19:0/2836332550 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8f2801a070 con 0x7f8f3c109b30 2024-04-25T11:05:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.619+0000 7f8f39ffb640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8f2c0afc60 0x7f8f2c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.619+0000 7f8f40bc1640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8f2c0afc60 0x7f8f2c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.620+0000 7f8f39ffb640 1 -- 172.21.15.19:0/2836332550 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f8f280deae0 con 0x7f8f3c109b30 2024-04-25T11:05:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.620+0000 7f8f40bc1640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8f2c0afc60 0x7f8f2c0b2170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005fd0 tx=0x7f8f3000b590 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:20.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.623+0000 7f8f39ffb640 1 -- 172.21.15.19:0/2836332550 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f3c19fd50 con 0x7f8f3c109b30 2024-04-25T11:05:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.811+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f3c10b6b0 con 0x7f8f3c109b30 2024-04-25T11:05:20.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.813+0000 7f8f39ffb640 1 -- 172.21.15.19:0/2836332550 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23125 (secure 0 0 0) 0x7f8f28009de0 con 0x7f8f3c109b30 2024-04-25T11:05:20.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:05:20.814 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T11:04:12.603369+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/48031724":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/2535542967":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/4264937365":"2024-04-26T10:54:29.901303+0000","172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3805408646":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6800/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8f2c0afc60 msgr2=0x7f8f2c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8f2c0afc60 0x7f8f2c0b2170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005fd0 tx=0x7f8f3000b590 comp rx=0 tx=0).stop 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c109b30 msgr2=0x7f8f3c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c109b30 0x7f8f3c19ac60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005d00 tx=0x7f8f2800dde0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 shutdown_connections 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f8f2c0afc60 0x7f8f2c0b2170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c109fd0 0x7f8f3c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f3c109b30 0x7f8f3c19ac60 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 --2- 172.21.15.19:0/2836332550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f3c104280 0x7f8f3c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 >> 172.21.15.19:0/2836332550 conn(0x7f8f3c0ff210 msgr2=0x7f8f3c1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 shutdown_connections 2024-04-25T11:05:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.815+0000 7f8f42e25640 1 -- 172.21.15.19:0/2836332550 wait complete. 2024-04-25T11:05:20.827 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:47:55.996597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:52:29.911135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:52:31.065421+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:05:20.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:05:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 -- 172.21.15.19:0/2154104844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1810e060 msgr2=0x7f6e18110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/2154104844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1810e060 0x7f6e18110550 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009ca0 tx=0x7f6e1002a9e0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 -- 172.21.15.19:0/2154104844 shutdown_connections 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/2154104844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1810e060 0x7f6e18110550 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/2154104844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e1810b6b0 0x7f6e1810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/2154104844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e18104c50 0x7f6e18105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 -- 172.21.15.19:0/2154104844 >> 172.21.15.19:0/2154104844 conn(0x7f6e18078100 msgr2=0x7f6e18078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 -- 172.21.15.19:0/2154104844 shutdown_connections 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.943+0000 7f6e1cccd640 1 -- 172.21.15.19:0/2154104844 wait complete. 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.944+0000 7f6e1cccd640 1 Processor -- start 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.944+0000 7f6e1cccd640 1 -- start start 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.944+0000 7f6e1cccd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e18104c50 0x7f6e1819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1cccd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e1810b6b0 0x7f6e1819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1cccd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e1810e060 0x7f6e1819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1cccd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e18113cc0 con 0x7f6e18104c50 2024-04-25T11:05:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1cccd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e18113b40 con 0x7f6e1810b6b0 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1cccd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e18113e40 con 0x7f6e1810e060 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e18104c50 0x7f6e1819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e18104c50 0x7f6e1819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41202/0 (socket says 172.21.15.19:41202) 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 -- 172.21.15.19:0/1650378274 learned_addr learned my addr 172.21.15.19:0/1650378274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e0ffff640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e1810b6b0 0x7f6e1819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e16d9d640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e1810e060 0x7f6e1819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 -- 172.21.15.19:0/1650378274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e1810e060 msgr2=0x7f6e1819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e1810e060 0x7f6e1819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 -- 172.21.15.19:0/1650378274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e1810b6b0 msgr2=0x7f6e1819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e1810b6b0 0x7f6e1819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 -- 172.21.15.19:0/1650378274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1819f540 con 0x7f6e18104c50 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1659c640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e18104c50 0x7f6e1819a6e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9f0 tx=0x7f6e0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e0f7fe640 1 -- 172.21.15.19:0/1650378274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800cd80 con 0x7f6e18104c50 2024-04-25T11:05:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e0f7fe640 1 -- 172.21.15.19:0/1650378274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e080108e0 con 0x7f6e18104c50 2024-04-25T11:05:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1819f830 con 0x7f6e18104c50 2024-04-25T11:05:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e0f7fe640 1 -- 172.21.15.19:0/1650378274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08018980 con 0x7f6e18104c50 2024-04-25T11:05:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.945+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e181a71b0 con 0x7f6e18104c50 2024-04-25T11:05:20.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.947+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1806afa0 con 0x7f6e18104c50 2024-04-25T11:05:20.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.948+0000 7f6e0f7fe640 1 -- 172.21.15.19:0/1650378274 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6e08010430 con 0x7f6e18104c50 2024-04-25T11:05:20.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.948+0000 7f6e0f7fe640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6e040afce0 0x7f6e040b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:20.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.949+0000 7f6e0ffff640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6e040afce0 0x7f6e040b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:20.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.949+0000 7f6e0f7fe640 1 -- 172.21.15.19:0/1650378274 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f6e080df440 con 0x7f6e18104c50 2024-04-25T11:05:20.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.949+0000 7f6e0ffff640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6e040afce0 0x7f6e040b21f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e1810e980 tx=0x7f6e00005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:20.953+0000 7f6e0f7fe640 1 -- 172.21.15.19:0/1650378274 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e080a7ce0 con 0x7f6e18104c50 2024-04-25T11:05:21.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.145+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e181a7380 con 0x7f6e18104c50 2024-04-25T11:05:21.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.145+0000 7f6e0f7fe640 1 -- 172.21.15.19:0/1650378274 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f6e080a75e0 con 0x7f6e18104c50 2024-04-25T11:05:21.146 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6e040afce0 msgr2=0x7f6e040b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6e040afce0 0x7f6e040b21f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e1810e980 tx=0x7f6e00005f90 comp rx=0 tx=0).stop 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e18104c50 msgr2=0x7f6e1819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e18104c50 0x7f6e1819a6e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9f0 tx=0x7f6e0800eeb0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 shutdown_connections 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f6e040afce0 0x7f6e040b21f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e1810e060 0x7f6e1819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e1810b6b0 0x7f6e1819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 --2- 172.21.15.19:0/1650378274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e18104c50 0x7f6e1819a6e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.147+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 >> 172.21.15.19:0/1650378274 conn(0x7f6e18078100 msgr2=0x7f6e18101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.148+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 shutdown_connections 2024-04-25T11:05:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.148+0000 7f6e1cccd640 1 -- 172.21.15.19:0/1650378274 wait complete. 2024-04-25T11:05:21.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:05:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/3672840510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 msgr2=0x7f1a9c1050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/3672840510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1050b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009900 tx=0x7f1a9802a480 comp rx=0 tx=0).stop 2024-04-25T11:05:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/3672840510 shutdown_connections 2024-04-25T11:05:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/3672840510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c10e0b0 0x7f1a9c110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/3672840510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9c10b700 0x7f1a9c10db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/3672840510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1050b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/3672840510 >> 172.21.15.19:0/3672840510 conn(0x7f1a9c078180 msgr2=0x7f1a9c0785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:21.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/3672840510 shutdown_connections 2024-04-25T11:05:21.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.275+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/3672840510 wait complete. 2024-04-25T11:05:21.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 Processor -- start 2024-04-25T11:05:21.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 -- start start 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c10b700 0x7f1a9c1a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9c10e0b0 0x7f1a9c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113cd0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113b50 con 0x7f1a9c10b700 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa3bc0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113e50 con 0x7f1a9c10e0b0 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.276+0000 7f1aa195c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41224/0 (socket says 172.21.15.19:41224) 2024-04-25T11:05:21.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 -- 172.21.15.19:0/1656996787 learned_addr learned my addr 172.21.15.19:0/1656996787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa115b640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c10b700 0x7f1a9c1a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa215d640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9c10e0b0 0x7f1a9c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 -- 172.21.15.19:0/1656996787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9c10e0b0 msgr2=0x7f1a9c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9c10e0b0 0x7f1a9c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 -- 172.21.15.19:0/1656996787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c10b700 msgr2=0x7f1a9c1a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c10b700 0x7f1a9c1a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 -- 172.21.15.19:0/1656996787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c19cea0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa195c640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1a25f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1a98005940 tx=0x7f1a980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1a8effd640 1 -- 172.21.15.19:0/1656996787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9803f030 con 0x7f1a9c104ca0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9c19d0d0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1a8effd640 1 -- 172.21.15.19:0/1656996787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a98003cb0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1a8effd640 1 -- 172.21.15.19:0/1656996787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98035070 con 0x7f1a9c104ca0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.277+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9c19d5c0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.279+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9c06baa0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.280+0000 7f1a8effd640 1 -- 172.21.15.19:0/1656996787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1a98042030 con 0x7f1a9c104ca0 2024-04-25T11:05:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.280+0000 7f1a8effd640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1a880afc60 0x7f1a880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.281+0000 7f1aa115b640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1a880afc60 0x7f1a880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.281+0000 7f1a8effd640 1 -- 172.21.15.19:0/1656996787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f1a980ff890 con 0x7f1a9c104ca0 2024-04-25T11:05:21.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.281+0000 7f1aa115b640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1a880afc60 0x7f1a880b2170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c19df10 tx=0x7f1a9000b410 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.285+0000 7f1a8effd640 1 -- 172.21.15.19:0/1656996787 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a980c7ae0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:21 smithi019 ceph-mon[29209]: pgmap v336: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:05:21.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:21 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2836332550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:05:21.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:21 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1650378274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:05:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.474+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1a9c1a9420 con 0x7f1a9c104ca0 2024-04-25T11:05:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.474+0000 7f1a8effd640 1 -- 172.21.15.19:0/1656996787 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f1a980d01b0 con 0x7f1a9c104ca0 2024-04-25T11:05:21.475 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.477+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1a880afc60 msgr2=0x7f1a880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.477+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1a880afc60 0x7f1a880b2170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c19df10 tx=0x7f1a9000b410 comp rx=0 tx=0).stop 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.477+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 msgr2=0x7f1a9c1a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.477+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1a25f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1a98005940 tx=0x7f1a980042f0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 shutdown_connections 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f1a880afc60 0x7f1a880b2170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9c10e0b0 0x7f1a9c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c10b700 0x7f1a9c1a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 --2- 172.21.15.19:0/1656996787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a9c104ca0 0x7f1a9c1a25f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 >> 172.21.15.19:0/1656996787 conn(0x7f1a9c078180 msgr2=0x7f1a9c10a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 shutdown_connections 2024-04-25T11:05:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.478+0000 7f1aa3bc0640 1 -- 172.21.15.19:0/1656996787 wait complete. 2024-04-25T11:05:21.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:05:21.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:21 smithi169 ceph-mon[33049]: pgmap v336: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:05:21.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:21 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2836332550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:05:21.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:21 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1650378274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:05:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:21 smithi112 ceph-mon[32799]: pgmap v336: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T11:05:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:21 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2836332550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:05:21.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:21 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1650378274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:05:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 -- 172.21.15.19:0/3657728791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204102af0 msgr2=0x7fe204102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/3657728791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204102af0 0x7fe204102f00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f402a410 comp rx=0 tx=0).stop 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 -- 172.21.15.19:0/3657728791 shutdown_connections 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/3657728791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe20410d380 0x7fe20410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/3657728791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204102af0 0x7fe204102f00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/3657728791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe204108a20 0x7fe2041025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 -- 172.21.15.19:0/3657728791 >> 172.21.15.19:0/3657728791 conn(0x7fe204078100 msgr2=0x7fe204078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 -- 172.21.15.19:0/3657728791 shutdown_connections 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.609+0000 7fe20b1e3640 1 -- 172.21.15.19:0/3657728791 wait complete. 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 Processor -- start 2024-04-25T11:05:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 -- start start 2024-04-25T11:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe204102af0 0x7fe204199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204108a20 0x7fe204199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe20410d380 0x7fe20419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe204112be0 con 0x7fe204108a20 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe204112a60 con 0x7fe204102af0 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe20b1e3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe204112d60 con 0x7fe20410d380 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204108a20 0x7fe204199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204108a20 0x7fe204199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41250/0 (socket says 172.21.15.19:41250) 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 -- 172.21.15.19:0/2869826489 learned_addr learned my addr 172.21.15.19:0/2869826489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe208f7f640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe204102af0 0x7fe204199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe209780640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe20410d380 0x7fe20419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 -- 172.21.15.19:0/2869826489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe20410d380 msgr2=0x7fe20419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe20410d380 0x7fe20419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 -- 172.21.15.19:0/2869826489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe204102af0 msgr2=0x7fe204199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe204102af0 0x7fe204199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.610+0000 7fe203fff640 1 -- 172.21.15.19:0/2869826489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20419e4c0 con 0x7fe204108a20 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.611+0000 7fe203fff640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204108a20 0x7fe204199bc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe1f402a930 tx=0x7fe1f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.611+0000 7fe201ffb640 1 -- 172.21.15.19:0/2869826489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4046070 con 0x7fe204108a20 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.611+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20419e750 con 0x7fe204108a20 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.611+0000 7fe201ffb640 1 -- 172.21.15.19:0/2869826489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f4003aa0 con 0x7fe204108a20 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.611+0000 7fe201ffb640 1 -- 172.21.15.19:0/2869826489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4035050 con 0x7fe204108a20 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.611+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20419ec90 con 0x7fe204108a20 2024-04-25T11:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.612+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe204103cd0 con 0x7fe204108a20 2024-04-25T11:05:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.614+0000 7fe201ffb640 1 -- 172.21.15.19:0/2869826489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe1f4049070 con 0x7fe204108a20 2024-04-25T11:05:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.614+0000 7fe201ffb640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe1f00afc60 0x7fe1f00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.614+0000 7fe201ffb640 1 -- 172.21.15.19:0/2869826489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fe1f4100b50 con 0x7fe204108a20 2024-04-25T11:05:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.614+0000 7fe208f7f640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe1f00afc60 0x7fe1f00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.615+0000 7fe208f7f640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe1f00afc60 0x7fe1f00b2170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0058c0 tx=0x7fe1ec00a2d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:21.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.619+0000 7fe201ffb640 1 -- 172.21.15.19:0/2869826489 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1f40c93f0 con 0x7fe204108a20 2024-04-25T11:05:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.808+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe204106a20 con 0x7fe204108a20 2024-04-25T11:05:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.808+0000 7fe201ffb640 1 -- 172.21.15.19:0/2869826489 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fe1f40c8cf0 con 0x7fe204108a20 2024-04-25T11:05:21.808 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe1f00afc60 msgr2=0x7fe1f00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe1f00afc60 0x7fe1f00b2170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0058c0 tx=0x7fe1ec00a2d0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204108a20 msgr2=0x7fe204199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204108a20 0x7fe204199bc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe1f402a930 tx=0x7fe1f4004270 comp rx=0 tx=0).stop 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 shutdown_connections 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fe1f00afc60 0x7fe1f00b2170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe20410d380 0x7fe20419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe204108a20 0x7fe204199bc0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 --2- 172.21.15.19:0/2869826489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe204102af0 0x7fe204199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 >> 172.21.15.19:0/2869826489 conn(0x7fe204078100 msgr2=0x7fe2041040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 shutdown_connections 2024-04-25T11:05:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.810+0000 7fe20b1e3640 1 -- 172.21.15.19:0/2869826489 wait complete. 2024-04-25T11:05:21.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 -- 172.21.15.19:0/2188470765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f588410b650 msgr2=0x7f588410da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 --2- 172.21.15.19:0/2188470765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f588410b650 0x7f588410da70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5874009900 tx=0x7f587402a410 comp rx=0 tx=0).stop 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 -- 172.21.15.19:0/2188470765 shutdown_connections 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 --2- 172.21.15.19:0/2188470765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f588410e040 0x7f5884110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 --2- 172.21.15.19:0/2188470765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f588410b650 0x7f588410da70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 --2- 172.21.15.19:0/2188470765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5884104bf0 0x7f5884105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 -- 172.21.15.19:0/2188470765 >> 172.21.15.19:0/2188470765 conn(0x7f5884078100 msgr2=0x7f5884078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 -- 172.21.15.19:0/2188470765 shutdown_connections 2024-04-25T11:05:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.936+0000 7f5889df3640 1 -- 172.21.15.19:0/2188470765 wait complete. 2024-04-25T11:05:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.937+0000 7f5889df3640 1 Processor -- start 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.937+0000 7f5889df3640 1 -- start start 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5889df3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5884104bf0 0x7f588419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5889df3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f588410b650 0x7f588419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5889df3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f588410e040 0x7f588419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5889df3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5884113ca0 con 0x7f5884104bf0 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5889df3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5884113b20 con 0x7f588410b650 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5889df3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5884113e20 con 0x7f588410e040 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5884104bf0 0x7f588419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5884104bf0 0x7f588419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41272/0 (socket says 172.21.15.19:41272) 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 -- 172.21.15.19:0/505560351 learned_addr learned my addr 172.21.15.19:0/505560351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5882ffd640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f588410b650 0x7f588419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f5883fff640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f588410e040 0x7f588419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 -- 172.21.15.19:0/505560351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f588410e040 msgr2=0x7f588419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f588410e040 0x7f588419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 -- 172.21.15.19:0/505560351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f588410b650 msgr2=0x7f588419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f588410b650 0x7f588419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 -- 172.21.15.19:0/505560351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588419f560 con 0x7f5884104bf0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.938+0000 7f58837fe640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5884104bf0 0x7f588419a700 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f587800e9f0 tx=0x7f587800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.939+0000 7f5880ff9640 1 -- 172.21.15.19:0/505560351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587800cd80 con 0x7f5884104bf0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.939+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588419f850 con 0x7f5884104bf0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.939+0000 7f5880ff9640 1 -- 172.21.15.19:0/505560351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58780108e0 con 0x7f5884104bf0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.939+0000 7f5880ff9640 1 -- 172.21.15.19:0/505560351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878018980 con 0x7f5884104bf0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.939+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58841a71d0 con 0x7f5884104bf0 2024-04-25T11:05:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.940+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588406afa0 con 0x7f5884104bf0 2024-04-25T11:05:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.941+0000 7f5880ff9640 1 -- 172.21.15.19:0/505560351 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5878010430 con 0x7f5884104bf0 2024-04-25T11:05:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.942+0000 7f5880ff9640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f586c0afc60 0x7f586c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.942+0000 7f5882ffd640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f586c0afc60 0x7f586c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.942+0000 7f5880ff9640 1 -- 172.21.15.19:0/505560351 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f58780df8c0 con 0x7f5884104bf0 2024-04-25T11:05:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.942+0000 7f5882ffd640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f586c0afc60 0x7f586c0b2170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5874031440 tx=0x7f5874009560 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:21.945+0000 7f5880ff9640 1 -- 172.21.15.19:0/505560351 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58780a8160 con 0x7f5884104bf0 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.146+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58841a73a0 con 0x7f5884104bf0 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.147+0000 7f5880ff9640 1 -- 172.21.15.19:0/505560351 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v88) ==== 81+0+937 (secure 0 0 0) 0x7f58780b01e0 con 0x7f5884104bf0 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1708221429 2024-04-26T10:54:29.901303+0000 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/48031724 2024-04-26T10:54:29.901303+0000 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2535542967 2024-04-26T10:54:29.901303+0000 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4264937365 2024-04-26T10:54:29.901303+0000 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1932674296 2024-04-26T10:45:29.902440+0000 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1932674296 2024-04-26T10:45:29.902440+0000 2024-04-25T11:05:22.147 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1427296 2024-04-26T10:45:29.902440+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3805408646 2024-04-26T10:54:29.901303+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3362794439 2024-04-26T10:45:29.902440+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/370155901 2024-04-26T10:45:01.574319+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1990434922 2024-04-26T10:45:01.574319+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/370155901 2024-04-26T10:45:01.574319+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/111580854 2024-04-26T10:45:01.574319+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1708221429 2024-04-26T10:54:29.901303+0000 2024-04-25T11:05:22.148 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/881789175 2024-04-26T10:45:01.574319+0000 2024-04-25T11:05:22.149 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1153623604 2024-04-26T10:45:29.902440+0000 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:listed 16 entries 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f586c0afc60 msgr2=0x7f586c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f586c0afc60 0x7f586c0b2170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5874031440 tx=0x7f5874009560 comp rx=0 tx=0).stop 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5884104bf0 msgr2=0x7f588419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5884104bf0 0x7f588419a700 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f587800e9f0 tx=0x7f587800eeb0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 shutdown_connections 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f586c0afc60 0x7f586c0b2170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f588410e040 0x7f588419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f588410b650 0x7f588419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 --2- 172.21.15.19:0/505560351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5884104bf0 0x7f588419a700 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 >> 172.21.15.19:0/505560351 conn(0x7f5884078100 msgr2=0x7f5884101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 shutdown_connections 2024-04-25T11:05:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.149+0000 7f5889df3640 1 -- 172.21.15.19:0/505560351 wait complete. 2024-04-25T11:05:22.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:05:22.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:05:22.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:05:22.208 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:05:22.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:05:22.208 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:05:22.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:05:22.246 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:05:22.302 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:05:22.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:05:22.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:05:22.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:05:22.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T11:05:22.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:22 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1656996787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:05:22.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:22 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/2869826489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:05:22.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:22 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/505560351' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:05:22.463 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T11:05:22.474 INFO:tasks.cephadm:Teardown begin 2024-04-25T11:05:22.474 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:05:22.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:05:22.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:22 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1656996787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:05:22.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:22 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/2869826489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:05:22.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:22 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/505560351' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:05:22.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:22 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1656996787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:05:22.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:22 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/2869826489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:05:22.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:22 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/505560351' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:05:22.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.615+0000 7f9fe55c0640 1 -- 172.21.15.19:0/801376817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 msgr2=0x7f9fe0101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.615+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/801376817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0101be0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009900 tx=0x7f9fd002a410 comp rx=0 tx=0).stop 2024-04-25T11:05:22.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 -- 172.21.15.19:0/801376817 shutdown_connections 2024-04-25T11:05:22.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/801376817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe01027d0 0x7f9fe0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/801376817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe0109fd0 0x7f9fe0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/801376817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0101be0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 -- 172.21.15.19:0/801376817 >> 172.21.15.19:0/801376817 conn(0x7f9fe00fd850 msgr2=0x7f9fe00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 -- 172.21.15.19:0/801376817 shutdown_connections 2024-04-25T11:05:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 -- 172.21.15.19:0/801376817 wait complete. 2024-04-25T11:05:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 Processor -- start 2024-04-25T11:05:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.616+0000 7f9fe55c0640 1 -- start start 2024-04-25T11:05:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fe55c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fe55c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe019a500 0x7f9fe019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fe55c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe019ecd0 0x7f9fe019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fe55c0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0113d20 con 0x7f9fe0109b30 2024-04-25T11:05:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fe55c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0113ba0 con 0x7f9fe019a500 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fe55c0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0113ea0 con 0x7f9fe019ecd0 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41290/0 (socket says 172.21.15.19:41290) 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 -- 172.21.15.19:0/543978792 learned_addr learned my addr 172.21.15.19:0/543978792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fde7fc640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe019a500 0x7f9fe019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdf7fe640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe019ecd0 0x7f9fe019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 -- 172.21.15.19:0/543978792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe019ecd0 msgr2=0x7f9fe019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe019ecd0 0x7f9fe019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 -- 172.21.15.19:0/543978792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe019a500 msgr2=0x7f9fe019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe019a500 0x7f9fe019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 -- 172.21.15.19:0/543978792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe019abb0 con 0x7f9fe0109b30 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fde7fc640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe019a500 0x7f9fe019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:05:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdf7fe640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe019ecd0 0x7f9fe019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:05:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fdeffd640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0199fc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006010 tx=0x7f9fd0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fc7fff640 1 -- 172.21.15.19:0/543978792 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd003f030 con 0x7f9fe0109b30 2024-04-25T11:05:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fc7fff640 1 -- 172.21.15.19:0/543978792 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd0003aa0 con 0x7f9fe0109b30 2024-04-25T11:05:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.617+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe01a6aa0 con 0x7f9fe0109b30 2024-04-25T11:05:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.618+0000 7f9fc7fff640 1 -- 172.21.15.19:0/543978792 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0035070 con 0x7f9fe0109b30 2024-04-25T11:05:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.618+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe01a7040 con 0x7f9fe0109b30 2024-04-25T11:05:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.618+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe006a190 con 0x7f9fe0109b30 2024-04-25T11:05:22.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.620+0000 7f9fc7fff640 1 -- 172.21.15.19:0/543978792 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9fd0042030 con 0x7f9fe0109b30 2024-04-25T11:05:22.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.621+0000 7f9fc7fff640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9fc80afce0 0x7f9fc80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.621+0000 7f9fde7fc640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9fc80afce0 0x7f9fc80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.621+0000 7f9fc7fff640 1 -- 172.21.15.19:0/543978792 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f9fd00ff8e0 con 0x7f9fe0109b30 2024-04-25T11:05:22.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.621+0000 7f9fde7fc640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9fc80afce0 0x7f9fc80b21f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005e10 tx=0x7f9fd4005da0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:22.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.624+0000 7f9fc7fff640 1 -- 172.21.15.19:0/543978792 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fd00c8180 con 0x7f9fe0109b30 2024-04-25T11:05:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.850+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fe019b2c0 con 0x7f9fe0109b30 2024-04-25T11:05:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.851+0000 7f9fc7fff640 1 -- 172.21.15.19:0/543978792 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f9fd00d0090 con 0x7f9fe0109b30 2024-04-25T11:05:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:05:22.852 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:05:22.852 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9fc80afce0 msgr2=0x7f9fc80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9fc80afce0 0x7f9fc80b21f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005e10 tx=0x7f9fd4005da0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 msgr2=0x7f9fe0199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0199fc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006010 tx=0x7f9fd0004060 comp rx=0 tx=0).stop 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 shutdown_connections 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9fc80afce0 0x7f9fc80b21f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe019ecd0 0x7f9fe019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe019a500 0x7f9fe019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 --2- 172.21.15.19:0/543978792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe0109b30 0x7f9fe0199fc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 >> 172.21.15.19:0/543978792 conn(0x7f9fe00fd850 msgr2=0x7f9fe0100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.853+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 shutdown_connections 2024-04-25T11:05:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.854+0000 7f9fe55c0640 1 -- 172.21.15.19:0/543978792 wait complete. 2024-04-25T11:05:22.865 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 -- 172.21.15.19:0/3998825794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109fd0 msgr2=0x7f90cc103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 --2- 172.21.15.19:0/3998825794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109fd0 0x7f90cc103c10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009900 tx=0x7f90c802a410 comp rx=0 tx=0).stop 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 -- 172.21.15.19:0/3998825794 shutdown_connections 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 --2- 172.21.15.19:0/3998825794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc104280 0x7f90cc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 --2- 172.21.15.19:0/3998825794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109fd0 0x7f90cc103c10 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 --2- 172.21.15.19:0/3998825794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc109b30 0x7f90cc1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 -- 172.21.15.19:0/3998825794 >> 172.21.15.19:0/3998825794 conn(0x7f90cc0ff210 msgr2=0x7f90cc101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 -- 172.21.15.19:0/3998825794 shutdown_connections 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.977+0000 7f90d44e7640 1 -- 172.21.15.19:0/3998825794 wait complete. 2024-04-25T11:05:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 Processor -- start 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 -- start start 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc104280 0x7f90cc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109b30 0x7f90cc19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc109fd0 0x7f90cc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc113d20 con 0x7f90cc109b30 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc113ba0 con 0x7f90cc104280 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d44e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc113ea0 con 0x7f90cc109fd0 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d1a82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109b30 0x7f90cc19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d1a82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109b30 0x7f90cc19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36958/0 (socket says 172.21.15.19:36958) 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.978+0000 7f90d1a82640 1 -- 172.21.15.19:0/1207303338 learned_addr learned my addr 172.21.15.19:0/1207303338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d2a84640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc109fd0 0x7f90cc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d2283640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc104280 0x7f90cc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d1a82640 1 -- 172.21.15.19:0/1207303338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc109fd0 msgr2=0x7f90cc19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d1a82640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc109fd0 0x7f90cc19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d1a82640 1 -- 172.21.15.19:0/1207303338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc104280 msgr2=0x7f90cc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d1a82640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc104280 0x7f90cc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d1a82640 1 -- 172.21.15.19:0/1207303338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc19f5e0 con 0x7f90cc109b30 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d2a84640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc109fd0 0x7f90cc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d2283640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc104280 0x7f90cc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d1a82640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109b30 0x7f90cc19ace0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f90c802a930 tx=0x7f90c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90c37fe640 1 -- 172.21.15.19:0/1207303338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c803f030 con 0x7f90cc109b30 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90c37fe640 1 -- 172.21.15.19:0/1207303338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c8003aa0 con 0x7f90cc109b30 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc19f870 con 0x7f90cc109b30 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90c37fe640 1 -- 172.21.15.19:0/1207303338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8035070 con 0x7f90cc109b30 2024-04-25T11:05:22.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.979+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc19fd60 con 0x7f90cc109b30 2024-04-25T11:05:22.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.980+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc06a190 con 0x7f90cc109b30 2024-04-25T11:05:22.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.982+0000 7f90c37fe640 1 -- 172.21.15.19:0/1207303338 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f90c8042030 con 0x7f90cc109b30 2024-04-25T11:05:22.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.983+0000 7f90c37fe640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f90b80afc60 0x7f90b80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:22.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.983+0000 7f90d2283640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f90b80afc60 0x7f90b80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:22.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.983+0000 7f90c37fe640 1 -- 172.21.15.19:0/1207303338 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f90c80ffc20 con 0x7f90cc109b30 2024-04-25T11:05:22.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.983+0000 7f90d2283640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f90b80afc60 0x7f90b80b2170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f90cc104810 tx=0x7f90bc0094b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:22.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:22.986+0000 7f90c37fe640 1 -- 172.21.15.19:0/1207303338 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90c80c84c0 con 0x7f90cc109b30 2024-04-25T11:05:23.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.215+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90cc1a7310 con 0x7f90cc109b30 2024-04-25T11:05:23.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.216+0000 7f90c37fe640 1 -- 172.21.15.19:0/1207303338 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11795 (secure 0 0 0) 0x7f90c80c7dc0 con 0x7f90cc109b30 2024-04-25T11:05:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:05:23.217 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/311381787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":311381787},{"type":"v1","addr":"172.21.15.19:6835","nonce":311381787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3384911547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3384911547},{"type":"v1","addr":"172.21.15.19:6837","nonce":3384911547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/154007495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":154007495},{"type":"v1","addr":"172.21.15.19:6839","nonce":154007495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/427683873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":427683873},{"type":"v1","addr":"172.21.15.19:6841","nonce":427683873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1523385162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1523385162},{"type":"v1","addr":"172.21.15.112:6833","nonce":1523385162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/894672787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":894672787},{"type":"v1","addr":"172.21.15.169:6833","nonce":894672787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/392627197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":392627197},{"type":"v1","addr":"172.21.15.112:6835","nonce":392627197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24488,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/820120755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":820120755},{"type":"v1","addr":"172.21.15.112:6837","nonce":820120755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1662431671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1662431671},{"type":"v1","addr":"172.21.15.169:6835","nonce":1662431671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/717205661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":717205661},{"type":"v1","addr":"172.21.15.169:6837","nonce":717205661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:52:32.071033+0000","modified":"2024-04-25T10:52:36.113733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6839/386384439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":386384439},{"type":"v1","addr":"172.21.15.169:6839","nonce":386384439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/850740995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":850740995},{"type":"v1","addr":"172.21.15.112:6839","nonce":850740995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-04-25T11:05:23.217 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f90b80afc60 msgr2=0x7f90b80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f90b80afc60 0x7f90b80b2170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f90cc104810 tx=0x7f90bc0094b0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109b30 msgr2=0x7f90cc19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109b30 0x7f90cc19ace0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f90c802a930 tx=0x7f90c8004270 comp rx=0 tx=0).stop 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 shutdown_connections 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f90b80afc60 0x7f90b80b2170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc109fd0 0x7f90cc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90cc109b30 0x7f90cc19ace0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 --2- 172.21.15.19:0/1207303338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc104280 0x7f90cc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 >> 172.21.15.19:0/1207303338 conn(0x7f90cc0ff210 msgr2=0x7f90cc10f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.218+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 shutdown_connections 2024-04-25T11:05:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.219+0000 7f90d44e7640 1 -- 172.21.15.19:0/1207303338 wait complete. 2024-04-25T11:05:23.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.342+0000 7f24f2fc7640 1 -- 172.21.15.19:0/962159670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec102b60 msgr2=0x7f24ec102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.342+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/962159670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec102b60 0x7f24ec102f70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009900 tx=0x7f24dc02a410 comp rx=0 tx=0).stop 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.342+0000 7f24f2fc7640 1 -- 172.21.15.19:0/962159670 shutdown_connections 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.342+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/962159670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec10d3a0 0x7f24ec10f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.342+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/962159670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec102b60 0x7f24ec102f70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.342+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/962159670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec108a40 0x7f24ec102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.342+0000 7f24f2fc7640 1 -- 172.21.15.19:0/962159670 >> 172.21.15.19:0/962159670 conn(0x7f24ec078040 msgr2=0x7f24ec078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 -- 172.21.15.19:0/962159670 shutdown_connections 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 -- 172.21.15.19:0/962159670 wait complete. 2024-04-25T11:05:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 Processor -- start 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 -- start start 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec102b60 0x7f24ec19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec108a40 0x7f24ec19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec10d3a0 0x7f24ec19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec112c00 con 0x7f24ec108a40 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec112a80 con 0x7f24ec102b60 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.343+0000 7f24f2fc7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec112d80 con 0x7f24ec10d3a0 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec108a40 0x7f24ec19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec108a40 0x7f24ec19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36970/0 (socket says 172.21.15.19:36970) 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 -- 172.21.15.19:0/4074557366 learned_addr learned my addr 172.21.15.19:0/4074557366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24f0d63640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec102b60 0x7f24ec19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24f1564640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec10d3a0 0x7f24ec19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 -- 172.21.15.19:0/4074557366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec10d3a0 msgr2=0x7f24ec19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec10d3a0 0x7f24ec19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 -- 172.21.15.19:0/4074557366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec102b60 msgr2=0x7f24ec19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec102b60 0x7f24ec19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 -- 172.21.15.19:0/4074557366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec1a6860 con 0x7f24ec108a40 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24f1564640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec10d3a0 0x7f24ec19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24f0d63640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec102b60 0x7f24ec19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24ebfff640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec108a40 0x7f24ec19ba60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f24dc02a930 tx=0x7f24dc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24e9ffb640 1 -- 172.21.15.19:0/4074557366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc03f030 con 0x7f24ec108a40 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24e9ffb640 1 -- 172.21.15.19:0/4074557366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc004490 con 0x7f24ec108a40 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24e9ffb640 1 -- 172.21.15.19:0/4074557366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc035070 con 0x7f24ec108a40 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec1a6af0 con 0x7f24ec108a40 2024-04-25T11:05:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.344+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec1a6f90 con 0x7f24ec108a40 2024-04-25T11:05:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.345+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ec06a190 con 0x7f24ec108a40 2024-04-25T11:05:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.348+0000 7f24e9ffb640 1 -- 172.21.15.19:0/4074557366 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f24dc042030 con 0x7f24ec108a40 2024-04-25T11:05:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.348+0000 7f24e9ffb640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f24d80afc60 0x7f24d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.349+0000 7f24f0d63640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f24d80afc60 0x7f24d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.349+0000 7f24e9ffb640 1 -- 172.21.15.19:0/4074557366 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f24dc0ff8d0 con 0x7f24ec108a40 2024-04-25T11:05:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.349+0000 7f24f0d63640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f24d80afc60 0x7f24d80b2170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f24ec103a40 tx=0x7f24d4005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.352+0000 7f24e9ffb640 1 -- 172.21.15.19:0/4074557366 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24dc0c8110 con 0x7f24ec108a40 2024-04-25T11:05:23.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:23 smithi169 ceph-mon[33049]: pgmap v337: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:05:23.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:23 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/543978792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:05:23.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.540+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24ec106970 con 0x7f24ec108a40 2024-04-25T11:05:23.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.542+0000 7f24e9ffb640 1 -- 172.21.15.19:0/4074557366 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23125 (secure 0 0 0) 0x7f24dc0c7a10 con 0x7f24ec108a40 2024-04-25T11:05:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:05:23.542 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"9e90b6fe-02f0-11ef-bc93-c7b262605968","created":"2024-04-25T10:44:27.527625+0000","modified":"2024-04-25T11:04:12.603369+0000","last_up_change":"2024-04-25T10:51:44.332136+0000","last_in_change":"2024-04-25T10:51:24.893533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:47:55.996597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:52:29.911135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:52:31.065421+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1a36b37-3389-4f73-873b-7dd279f1701a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6803","nonce":2420046240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6805","nonce":2420046240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6809","nonce":2420046240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2420046240},{"type":"v1","addr":"172.21.15.19:6807","nonce":2420046240}]},"public_addr":"172.21.15.19:6803/2420046240","cluster_addr":"172.21.15.19:6805/2420046240","heartbeat_back_addr":"172.21.15.19:6809/2420046240","heartbeat_front_addr":"172.21.15.19:6807/2420046240","state":["exists","up"]},{"osd":1,"uuid":"d41a19a6-e1ab-4d69-87da-3ce650ec50c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6801","nonce":3663453616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6803","nonce":3663453616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6807","nonce":3663453616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3663453616},{"type":"v1","addr":"172.21.15.112:6805","nonce":3663453616}]},"public_addr":"172.21.15.112:6801/3663453616","cluster_addr":"172.21.15.112:6803/3663453616","heartbeat_back_addr":"172.21.15.112:6807/3663453616","heartbeat_front_addr":"172.21.15.112:6805/3663453616","state":["exists","up"]},{"osd":2,"uuid":"d74cbdc7-0138-4cd7-92ec-f4a16a3edd1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6801","nonce":3608720428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6803","nonce":3608720428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6807","nonce":3608720428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3608720428},{"type":"v1","addr":"172.21.15.169:6805","nonce":3608720428}]},"public_addr":"172.21.15.169:6801/3608720428","cluster_addr":"172.21.15.169:6803/3608720428","heartbeat_back_addr":"172.21.15.169:6807/3608720428","heartbeat_front_addr":"172.21.15.169:6805/3608720428","state":["exists","up"]},{"osd":3,"uuid":"efc21e6a-f4d3-4abd-aee6-b6a8533142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6811","nonce":930248439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6813","nonce":930248439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6817","nonce":930248439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":930248439},{"type":"v1","addr":"172.21.15.19:6815","nonce":930248439}]},"public_addr":"172.21.15.19:6811/930248439","cluster_addr":"172.21.15.19:6813/930248439","heartbeat_back_addr":"172.21.15.19:6817/930248439","heartbeat_front_addr":"172.21.15.19:6815/930248439","state":["exists","up"]},{"osd":4,"uuid":"a1ec5ae4-0515-4c9d-a701-7ea0e010534f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6809","nonce":3068928119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6811","nonce":3068928119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6815","nonce":3068928119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3068928119},{"type":"v1","addr":"172.21.15.112:6813","nonce":3068928119}]},"public_addr":"172.21.15.112:6809/3068928119","cluster_addr":"172.21.15.112:6811/3068928119","heartbeat_back_addr":"172.21.15.112:6815/3068928119","heartbeat_front_addr":"172.21.15.112:6813/3068928119","state":["exists","up"]},{"osd":5,"uuid":"6c271d7f-d82b-4849-a765-f0d0de0ad3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6809","nonce":3010722907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6811","nonce":3010722907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6815","nonce":3010722907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3010722907},{"type":"v1","addr":"172.21.15.169:6813","nonce":3010722907}]},"public_addr":"172.21.15.169:6809/3010722907","cluster_addr":"172.21.15.169:6811/3010722907","heartbeat_back_addr":"172.21.15.169:6815/3010722907","heartbeat_front_addr":"172.21.15.169:6813/3010722907","state":["exists","up"]},{"osd":6,"uuid":"5a24aa3b-f969-467b-8fdf-f0550d37a071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6819","nonce":2613393193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6821","nonce":2613393193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6825","nonce":2613393193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2613393193},{"type":"v1","addr":"172.21.15.19:6823","nonce":2613393193}]},"public_addr":"172.21.15.19:6819/2613393193","cluster_addr":"172.21.15.19:6821/2613393193","heartbeat_back_addr":"172.21.15.19:6825/2613393193","heartbeat_front_addr":"172.21.15.19:6823/2613393193","state":["exists","up"]},{"osd":7,"uuid":"1325203b-15e0-415c-a705-0e6b38ef714a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6817","nonce":2341909802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6819","nonce":2341909802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6823","nonce":2341909802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2341909802},{"type":"v1","addr":"172.21.15.112:6821","nonce":2341909802}]},"public_addr":"172.21.15.112:6817/2341909802","cluster_addr":"172.21.15.112:6819/2341909802","heartbeat_back_addr":"172.21.15.112:6823/2341909802","heartbeat_front_addr":"172.21.15.112:6821/2341909802","state":["exists","up"]},{"osd":8,"uuid":"45140686-3fd7-44dd-a246-d67730c2b8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6817","nonce":4055159566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6819","nonce":4055159566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6823","nonce":4055159566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4055159566},{"type":"v1","addr":"172.21.15.169:6821","nonce":4055159566}]},"public_addr":"172.21.15.169:6817/4055159566","cluster_addr":"172.21.15.169:6819/4055159566","heartbeat_back_addr":"172.21.15.169:6823/4055159566","heartbeat_front_addr":"172.21.15.169:6821/4055159566","state":["exists","up"]},{"osd":9,"uuid":"1a8af5c3-8fc8-4f98-9fbc-b716ab68ac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6827","nonce":920660159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6829","nonce":920660159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6833","nonce":920660159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":920660159},{"type":"v1","addr":"172.21.15.19:6831","nonce":920660159}]},"public_addr":"172.21.15.19:6827/920660159","cluster_addr":"172.21.15.19:6829/920660159","heartbeat_back_addr":"172.21.15.19:6833/920660159","heartbeat_front_addr":"172.21.15.19:6831/920660159","state":["exists","up"]},{"osd":10,"uuid":"6301d36c-b3e4-4840-bde6-833e2da34788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6825","nonce":2703835627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6827","nonce":2703835627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6831","nonce":2703835627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2703835627},{"type":"v1","addr":"172.21.15.112:6829","nonce":2703835627}]},"public_addr":"172.21.15.112:6825/2703835627","cluster_addr":"172.21.15.112:6827/2703835627","heartbeat_back_addr":"172.21.15.112:6831/2703835627","heartbeat_front_addr":"172.21.15.112:6829/2703835627","state":["exists","up"]},{"osd":11,"uuid":"9479e8e3-d6cf-4826-8411-9c6801ec4fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6825","nonce":2887715484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6827","nonce":2887715484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6831","nonce":2887715484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2887715484},{"type":"v1","addr":"172.21.15.169:6829","nonce":2887715484}]},"public_addr":"172.21.15.169:6825/2887715484","cluster_addr":"172.21.15.169:6827/2887715484","heartbeat_back_addr":"172.21.15.169:6831/2887715484","heartbeat_front_addr":"172.21.15.169:6829/2887715484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:04.048632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:28.646070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:47:53.426710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:18.957592+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:48:43.841479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:08.550857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:34.463513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:49:59.831058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:25.066693+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:50:52.054006+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:17.545484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:51:43.087300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6801/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/48031724":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/2535542967":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/4264937365":"2024-04-26T10:54:29.901303+0000","172.21.15.19:6801/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/1932674296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/1427296":"2024-04-26T10:45:29.902440+0000","172.21.15.19:0/3805408646":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/3362794439":"2024-04-26T10:45:29.902440+0000","172.21.15.19:6800/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1990434922":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6801/370155901":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/111580854":"2024-04-26T10:45:01.574319+0000","172.21.15.19:6800/1708221429":"2024-04-26T10:54:29.901303+0000","172.21.15.19:0/881789175":"2024-04-26T10:45:01.574319+0000","172.21.15.19:0/1153623604":"2024-04-26T10:45:29.902440+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:05:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f24d80afc60 msgr2=0x7f24d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f24d80afc60 0x7f24d80b2170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f24ec103a40 tx=0x7f24d4005f90 comp rx=0 tx=0).stop 2024-04-25T11:05:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec108a40 msgr2=0x7f24ec19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec108a40 0x7f24ec19ba60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f24dc02a930 tx=0x7f24dc031a20 comp rx=0 tx=0).stop 2024-04-25T11:05:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 shutdown_connections 2024-04-25T11:05:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f24d80afc60 0x7f24d80b2170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec10d3a0 0x7f24ec19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24ec108a40 0x7f24ec19ba60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 --2- 172.21.15.19:0/4074557366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec102b60 0x7f24ec19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 >> 172.21.15.19:0/4074557366 conn(0x7f24ec078040 msgr2=0x7f24ec1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 shutdown_connections 2024-04-25T11:05:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.544+0000 7f24f2fc7640 1 -- 172.21.15.19:0/4074557366 wait complete. 2024-04-25T11:05:23.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:23 smithi019 ceph-mon[29209]: pgmap v337: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:05:23.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:23 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/543978792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:05:23.556 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:05:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:23 smithi112 ceph-mon[32799]: pgmap v337: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T11:05:23.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:23 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/543978792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.669+0000 7f9bb074d640 1 -- 172.21.15.19:0/3339188660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 msgr2=0x7f9ba8102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.669+0000 7f9bb074d640 1 --2- 172.21.15.19:0/3339188660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba8102b00 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009900 tx=0x7f9b9802a410 comp rx=0 tx=0).stop 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 -- 172.21.15.19:0/3339188660 shutdown_connections 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 --2- 172.21.15.19:0/3339188660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8103170 0x7f9ba810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 --2- 172.21.15.19:0/3339188660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba8102b00 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 --2- 172.21.15.19:0/3339188660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ba8108a10 0x7f9ba81025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 -- 172.21.15.19:0/3339188660 >> 172.21.15.19:0/3339188660 conn(0x7f9ba8078150 msgr2=0x7f9ba8078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 -- 172.21.15.19:0/3339188660 shutdown_connections 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 -- 172.21.15.19:0/3339188660 wait complete. 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 Processor -- start 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 -- start start 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.670+0000 7f9bb074d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8103170 0x7f9ba819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bb074d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ba8108a10 0x7f9ba819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bb074d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bb074d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8112c10 con 0x7f9ba8108eb0 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bb074d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8112a90 con 0x7f9ba8108a10 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bb074d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8112d90 con 0x7f9ba8103170 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba819bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36974/0 (socket says 172.21.15.19:36974) 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 -- 172.21.15.19:0/414542436 learned_addr learned my addr 172.21.15.19:0/414542436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bae4e9640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8103170 0x7f9ba819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9badce8640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ba8108a10 0x7f9ba819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 -- 172.21.15.19:0/414542436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8103170 msgr2=0x7f9ba819b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8103170 0x7f9ba819b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 -- 172.21.15.19:0/414542436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ba8108a10 msgr2=0x7f9ba819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ba8108a10 0x7f9ba819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 -- 172.21.15.19:0/414542436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba81a6890 con 0x7f9ba8108eb0 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9baecea640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba819bff0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000eaa0 tx=0x7f9ba000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9b9f7fe640 1 -- 172.21.15.19:0/414542436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000cdb0 con 0x7f9ba8108eb0 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9b9f7fe640 1 -- 172.21.15.19:0/414542436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba0005020 con 0x7f9ba8108eb0 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba81a6b80 con 0x7f9ba8108eb0 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9b9f7fe640 1 -- 172.21.15.19:0/414542436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00116d0 con 0x7f9ba8108eb0 2024-04-25T11:05:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.671+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba81a70a0 con 0x7f9ba8108eb0 2024-04-25T11:05:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.672+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba806a160 con 0x7f9ba8108eb0 2024-04-25T11:05:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.674+0000 7f9b9f7fe640 1 -- 172.21.15.19:0/414542436 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9ba00051d0 con 0x7f9ba8108eb0 2024-04-25T11:05:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.674+0000 7f9b9f7fe640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9b940afce0 0x7f9b940b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.675+0000 7f9b9f7fe640 1 -- 172.21.15.19:0/414542436 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f9ba00df530 con 0x7f9ba8108eb0 2024-04-25T11:05:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.675+0000 7f9bae4e9640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9b940afce0 0x7f9b940b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.675+0000 7f9bae4e9640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9b940afce0 0x7f9b940b21f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ba806c470 tx=0x7f9ba4005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:23.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.678+0000 7f9b9f7fe640 1 -- 172.21.15.19:0/414542436 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ba0016030 con 0x7f9ba8108eb0 2024-04-25T11:05:23.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.903+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9ba80753e0 con 0x7f9ba8108eb0 2024-04-25T11:05:23.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.905+0000 7f9b9f7fe640 1 -- 172.21.15.19:0/414542436 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f9ba80753e0 con 0x7f9ba8108eb0 2024-04-25T11:05:23.905 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9b940afce0 msgr2=0x7f9b940b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9b940afce0 0x7f9b940b21f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ba806c470 tx=0x7f9ba4005f90 comp rx=0 tx=0).stop 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 msgr2=0x7f9ba819bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba819bff0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000eaa0 tx=0x7f9ba000ef60 comp rx=0 tx=0).stop 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 shutdown_connections 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f9b940afce0 0x7f9b940b21f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba8108eb0 0x7f9ba819bff0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ba8108a10 0x7f9ba819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 --2- 172.21.15.19:0/414542436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8103170 0x7f9ba819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 >> 172.21.15.19:0/414542436 conn(0x7f9ba8078150 msgr2=0x7f9ba81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 shutdown_connections 2024-04-25T11:05:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:23.907+0000 7f9bb074d640 1 -- 172.21.15.19:0/414542436 wait complete. 2024-04-25T11:05:23.920 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:05:24.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 -- 172.21.15.19:0/1164273167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 msgr2=0x7f5764103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:24.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 --2- 172.21.15.19:0/1164273167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f5764103bf0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5758009900 tx=0x7f575802a410 comp rx=0 tx=0).stop 2024-04-25T11:05:24.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 -- 172.21.15.19:0/1164273167 shutdown_connections 2024-04-25T11:05:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 --2- 172.21.15.19:0/1164273167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764104260 0x7f5764110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 --2- 172.21.15.19:0/1164273167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f5764103bf0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 --2- 172.21.15.19:0/1164273167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764109b10 0x7f57641036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 -- 172.21.15.19:0/1164273167 >> 172.21.15.19:0/1164273167 conn(0x7f57640ff210 msgr2=0x7f5764101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 -- 172.21.15.19:0/1164273167 shutdown_connections 2024-04-25T11:05:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.034+0000 7f576a615640 1 -- 172.21.15.19:0/1164273167 wait complete. 2024-04-25T11:05:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.035+0000 7f576a615640 1 Processor -- start 2024-04-25T11:05:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.035+0000 7f576a615640 1 -- start start 2024-04-25T11:05:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f576a615640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764104260 0x7f576419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f576a615640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764109b10 0x7f576419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f576a615640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f576419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f576a615640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5764113d00 con 0x7f5764109fb0 2024-04-25T11:05:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f576a615640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5764113b80 con 0x7f5764104260 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f576a615640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5764113e80 con 0x7f5764109b10 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f576419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f576419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36990/0 (socket says 172.21.15.19:36990) 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 -- 172.21.15.19:0/4215201351 learned_addr learned my addr 172.21.15.19:0/4215201351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5763fff640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764104260 0x7f576419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f57637fe640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764109b10 0x7f576419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 -- 172.21.15.19:0/4215201351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764109b10 msgr2=0x7f576419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764109b10 0x7f576419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 -- 172.21.15.19:0/4215201351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764104260 msgr2=0x7f576419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:24.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764104260 0x7f576419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 -- 172.21.15.19:0/4215201351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576419f640 con 0x7f5764109fb0 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5763fff640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764104260 0x7f576419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f5768bb2640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f576419f000 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f575c00eaa0 tx=0x7f575c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.036+0000 7f57617fa640 1 -- 172.21.15.19:0/4215201351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c00cdb0 con 0x7f5764109fb0 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.037+0000 7f57617fa640 1 -- 172.21.15.19:0/4215201351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f575c005020 con 0x7f5764109fb0 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.037+0000 7f57617fa640 1 -- 172.21.15.19:0/4215201351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c0116d0 con 0x7f5764109fb0 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.037+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576419f930 con 0x7f5764109fb0 2024-04-25T11:05:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.037+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57641a72b0 con 0x7f5764109fb0 2024-04-25T11:05:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.038+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576406a160 con 0x7f5764109fb0 2024-04-25T11:05:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.039+0000 7f57617fa640 1 -- 172.21.15.19:0/4215201351 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f575c0051d0 con 0x7f5764109fb0 2024-04-25T11:05:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.040+0000 7f57617fa640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f574c0afce0 0x7f574c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.040+0000 7f5763fff640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f574c0afce0 0x7f574c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.040+0000 7f57617fa640 1 -- 172.21.15.19:0/4215201351 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8555+0+0 (secure 0 0 0) 0x7f575c0df5f0 con 0x7f5764109fb0 2024-04-25T11:05:24.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.040+0000 7f5763fff640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f574c0afce0 0x7f574c0b21f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5754005e00 tx=0x7f5754005c70 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:24.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.043+0000 7f57617fa640 1 -- 172.21.15.19:0/4215201351 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f575c016030 con 0x7f5764109fb0 2024-04-25T11:05:24.220 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a[29205]: 2024-04-25T11:05:23.903+0000 7f39a7eec700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1207303338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4074557366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/414542436' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/414542436' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T11:05:24.251 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:24 smithi169 ceph-mon[33049]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T11:05:24.252 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:05:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i[63909]: ignoring --setuser ceph since I am not root 2024-04-25T11:05:24.252 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:05:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i[63909]: ignoring --setgroup ceph since I am not root 2024-04-25T11:05:24.252 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:05:23 smithi169 ceph-mds[63913]: -- 172.21.15.169:0/694257680 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559dfb177a40 con 0x559dfb1bf600 2024-04-25T11:05:24.252 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:05:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i[63909]: starting mds.i at 2024-04-25T11:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.273+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f57641a7480 con 0x7f5764109fb0 2024-04-25T11:05:24.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:05:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b[62588]: ignoring --setuser ceph since I am not root 2024-04-25T11:05:24.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:05:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b[62588]: ignoring --setgroup ceph since I am not root 2024-04-25T11:05:24.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:05:23 smithi112 ceph-mds[62592]: -- 172.21.15.112:0/3590826848 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588d049fa40 con 0x5588d04e7180 2024-04-25T11:05:24.309 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:05:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b[62588]: starting mds.b at 2024-04-25T11:05:24.309 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1207303338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:05:24.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4074557366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:05:24.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/414542436' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:05:24.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:05:24.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:05:24.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/414542436' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:05:24.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T11:05:24.311 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:24 smithi112 ceph-mon[32799]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T11:05:24.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1207303338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:05:24.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4074557366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:05:24.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/414542436' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:05:24.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:05:24.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:05:24.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/414542436' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:05:24.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T11:05:24.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:24 smithi019 ceph-mon[29209]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T11:05:24.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.911+0000 7f57617fa640 1 -- 172.21.15.19:0/4215201351 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f575c0a7ef0 con 0x7f5764109fb0 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f574c0afce0 msgr2=0x7f574c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f574c0afce0 0x7f574c0b21f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5754005e00 tx=0x7f5754005c70 comp rx=0 tx=0).stop 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 msgr2=0x7f576419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f576419f000 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f575c00eaa0 tx=0x7f575c00ef60 comp rx=0 tx=0).stop 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 shutdown_connections 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f574c0afce0 0x7f574c0b21f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5764109fb0 0x7f576419f000 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764109b10 0x7f576419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 --2- 172.21.15.19:0/4215201351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764104260 0x7f576419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 >> 172.21.15.19:0/4215201351 conn(0x7f57640ff210 msgr2=0x7f576410f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 shutdown_connections 2024-04-25T11:05:24.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:24.914+0000 7f576a615640 1 -- 172.21.15.19:0/4215201351 wait complete. 2024-04-25T11:05:24.930 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:05:25.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.053+0000 7f22a3be3640 1 -- 172.21.15.19:0/2169628087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109b10 msgr2=0x7f229c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:25.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.053+0000 7f22a3be3640 1 --2- 172.21.15.19:0/2169628087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109b10 0x7f229c1036b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2294009900 tx=0x7f229402a410 comp rx=0 tx=0).stop 2024-04-25T11:05:25.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.053+0000 7f22a3be3640 1 -- 172.21.15.19:0/2169628087 shutdown_connections 2024-04-25T11:05:25.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.053+0000 7f22a3be3640 1 --2- 172.21.15.19:0/2169628087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f229c104260 0x7f229c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.053+0000 7f22a3be3640 1 --2- 172.21.15.19:0/2169628087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f229c109fb0 0x7f229c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.053+0000 7f22a3be3640 1 --2- 172.21.15.19:0/2169628087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109b10 0x7f229c1036b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.053+0000 7f22a3be3640 1 -- 172.21.15.19:0/2169628087 >> 172.21.15.19:0/2169628087 conn(0x7f229c0ff1d0 msgr2=0x7f229c101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.054+0000 7f22a3be3640 1 -- 172.21.15.19:0/2169628087 shutdown_connections 2024-04-25T11:05:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.054+0000 7f22a3be3640 1 -- 172.21.15.19:0/2169628087 wait complete. 2024-04-25T11:05:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.054+0000 7f22a3be3640 1 Processor -- start 2024-04-25T11:05:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.054+0000 7f22a3be3640 1 -- start start 2024-04-25T11:05:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f229c104260 0x7f229c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f229c109b10 0x7f229c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109fb0 0x7f229c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f229c113d00 con 0x7f229c109fb0 2024-04-25T11:05:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f229c113b80 con 0x7f229c104260 2024-04-25T11:05:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f229c113e80 con 0x7f229c109b10 2024-04-25T11:05:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109fb0 0x7f229c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109fb0 0x7f229c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36998/0 (socket says 172.21.15.19:36998) 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 -- 172.21.15.19:0/3122897813 learned_addr learned my addr 172.21.15.19:0/3122897813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a197f640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f229c104260 0x7f229c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a117e640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f229c109b10 0x7f229c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 -- 172.21.15.19:0/3122897813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f229c109b10 msgr2=0x7f229c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f229c109b10 0x7f229c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 -- 172.21.15.19:0/3122897813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f229c104260 msgr2=0x7f229c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f229c104260 0x7f229c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 -- 172.21.15.19:0/3122897813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c19f600 con 0x7f229c109fb0 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a197f640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f229c104260 0x7f229c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a2180640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109fb0 0x7f229c19efc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2298009fd0 tx=0x7f229800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f2292ffd640 1 -- 172.21.15.19:0/3122897813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298017070 con 0x7f229c109fb0 2024-04-25T11:05:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c19f8f0 con 0x7f229c109fb0 2024-04-25T11:05:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f2292ffd640 1 -- 172.21.15.19:0/3122897813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2298003e20 con 0x7f229c109fb0 2024-04-25T11:05:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f2292ffd640 1 -- 172.21.15.19:0/3122897813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298010460 con 0x7f229c109fb0 2024-04-25T11:05:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.055+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1a7270 con 0x7f229c109fb0 2024-04-25T11:05:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.056+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c104df0 con 0x7f229c109fb0 2024-04-25T11:05:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.059+0000 7f2292ffd640 1 -- 172.21.15.19:0/3122897813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f2298003970 con 0x7f229c109fb0 2024-04-25T11:05:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.059+0000 7f2292ffd640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f22880afc60 0x7f22880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.059+0000 7f22a197f640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f22880afc60 0x7f22880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.059+0000 7f2292ffd640 1 -- 172.21.15.19:0/3122897813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8555+0+0 (secure 0 0 0) 0x7f22980deea0 con 0x7f229c109fb0 2024-04-25T11:05:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.060+0000 7f22a197f640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f22880afc60 0x7f22880b2170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2294000c00 tx=0x7f2294032cc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.063+0000 7f2292ffd640 1 -- 172.21.15.19:0/3122897813 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22980a7720 con 0x7f229c109fb0 2024-04-25T11:05:25.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:25.251+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f229c06a160 con 0x7f229c109fb0 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: pgmap v339: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4215201351' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: Cluster is now healthy 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/4215201351' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.169:6838/343397866,v1:172.21.15.169:6839/343397866] up:boot 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: mds.? [v2:172.21.15.112:6838/2877478735,v1:172.21.15.112:6839/2877478735] up:boot 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: fsmap 12 up:standby 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:05:25.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:25 smithi169 ceph-mon[33049]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:05:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: pgmap v339: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T11:05:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4215201351' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:05:25.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: Cluster is now healthy 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/4215201351' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.169:6838/343397866,v1:172.21.15.169:6839/343397866] up:boot 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: mds.? [v2:172.21.15.112:6838/2877478735,v1:172.21.15.112:6839/2877478735] up:boot 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: fsmap 12 up:standby 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:05:25.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:25 smithi112 ceph-mon[32799]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:05:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: pgmap v339: 97 pgs: 97 active+clean; 11 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T11:05:25.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4215201351' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: Cluster is now healthy 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/4215201351' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.169:6838/343397866,v1:172.21.15.169:6839/343397866] up:boot 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: mds.? [v2:172.21.15.112:6838/2877478735,v1:172.21.15.112:6839/2877478735] up:boot 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: fsmap 12 up:standby 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:05:25.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:25 smithi019 ceph-mon[29209]: from='mgr.15210 172.21.15.19:0/2281730051' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:05:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.228+0000 7f2292ffd640 1 -- 172.21.15.19:0/3122897813 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7f229c06a160 con 0x7f229c109fb0 2024-04-25T11:05:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.231+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f22880afc60 msgr2=0x7f22880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.231+0000 7f22a3be3640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f22880afc60 0x7f22880b2170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2294000c00 tx=0x7f2294032cc0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109fb0 msgr2=0x7f229c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109fb0 0x7f229c19efc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2298009fd0 tx=0x7f229800b260 comp rx=0 tx=0).stop 2024-04-25T11:05:26.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 shutdown_connections 2024-04-25T11:05:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7f22880afc60 0x7f22880b2170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f229c109fb0 0x7f229c19efc0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f229c109b10 0x7f229c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 --2- 172.21.15.19:0/3122897813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f229c104260 0x7f229c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 >> 172.21.15.19:0/3122897813 conn(0x7f229c0ff1d0 msgr2=0x7f229c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 shutdown_connections 2024-04-25T11:05:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.232+0000 7f22a3be3640 1 -- 172.21.15.19:0/3122897813 wait complete. 2024-04-25T11:05:26.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 -- 172.21.15.19:0/2935688902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf0109fb0 msgr2=0x7fdcf0103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/2935688902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf0109fb0 0x7fdcf0103c40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009900 tx=0x7fdce402a410 comp rx=0 tx=0).stop 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 -- 172.21.15.19:0/2935688902 shutdown_connections 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/2935688902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcf01042b0 0x7fdcf0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/2935688902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf0109fb0 0x7fdcf0103c40 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/2935688902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcf0109b10 0x7fdcf0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 -- 172.21.15.19:0/2935688902 >> 172.21.15.19:0/2935688902 conn(0x7fdcf00ff360 msgr2=0x7fdcf0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 -- 172.21.15.19:0/2935688902 shutdown_connections 2024-04-25T11:05:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 -- 172.21.15.19:0/2935688902 wait complete. 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.370+0000 7fdcf71a8640 1 Processor -- start 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf71a8640 1 -- start start 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf71a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf01042b0 0x7fdcf019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf71a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcf0109b10 0x7fdcf019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf71a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcf0109fb0 0x7fdcf019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf71a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0113d80 con 0x7fdcf01042b0 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf71a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0113c00 con 0x7fdcf0109b10 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf71a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0113f00 con 0x7fdcf0109fb0 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf01042b0 0x7fdcf019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf01042b0 0x7fdcf019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37024/0 (socket says 172.21.15.19:37024) 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 -- 172.21.15.19:0/1273091577 learned_addr learned my addr 172.21.15.19:0/1273091577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:05:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf5745640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcf0109fb0 0x7fdcf019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdceffff640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcf0109b10 0x7fdcf019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 -- 172.21.15.19:0/1273091577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcf0109fb0 msgr2=0x7fdcf019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcf0109fb0 0x7fdcf019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 -- 172.21.15.19:0/1273091577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcf0109b10 msgr2=0x7fdcf019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcf0109b10 0x7fdcf019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 -- 172.21.15.19:0/1273091577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf019f640 con 0x7fdcf01042b0 2024-04-25T11:05:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.371+0000 7fdcf4f44640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf01042b0 0x7fdcf019a7e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdce000b450 tx=0x7fdce000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.372+0000 7fdcedffb640 1 -- 172.21.15.19:0/1273091577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0015020 con 0x7fdcf01042b0 2024-04-25T11:05:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.372+0000 7fdcedffb640 1 -- 172.21.15.19:0/1273091577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdce0011d40 con 0x7fdcf01042b0 2024-04-25T11:05:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.372+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf019f930 con 0x7fdcf01042b0 2024-04-25T11:05:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.372+0000 7fdcedffb640 1 -- 172.21.15.19:0/1273091577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce000f340 con 0x7fdcf01042b0 2024-04-25T11:05:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.372+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf01a72b0 con 0x7fdcf01042b0 2024-04-25T11:05:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.373+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf006afa0 con 0x7fdcf01042b0 2024-04-25T11:05:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.374+0000 7fdcedffb640 1 -- 172.21.15.19:0/1273091577 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fdce0018020 con 0x7fdcf01042b0 2024-04-25T11:05:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.375+0000 7fdcedffb640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fdcd80afce0 0x7fdcd80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:05:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.375+0000 7fdceffff640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fdcd80afce0 0x7fdcd80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:05:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.375+0000 7fdcedffb640 1 -- 172.21.15.19:0/1273091577 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8059+0+0 (secure 0 0 0) 0x7fdce00de710 con 0x7fdcf01042b0 2024-04-25T11:05:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.376+0000 7fdceffff640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fdcd80afce0 0x7fdcd80b21f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009230 tx=0x7fdce4031400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:05:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.380+0000 7fdcedffb640 1 -- 172.21.15.19:0/1273091577 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdce001f020 con 0x7fdcf01042b0 2024-04-25T11:05:26.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:26 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3122897813' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:05:26.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:26 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3122897813' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:05:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:26.574+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdcf01a7480 con 0x7fdcf01042b0 2024-04-25T11:05:26.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:26 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3122897813' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:05:27.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.230+0000 7fdcedffb640 1 -- 172.21.15.19:0/1273091577 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7fdce000d040 con 0x7fdcf01042b0 2024-04-25T11:05:27.230 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.233+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fdcd80afce0 msgr2=0x7fdcd80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.233+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fdcd80afce0 0x7fdcd80b21f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009230 tx=0x7fdce4031400 comp rx=0 tx=0).stop 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.233+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf01042b0 msgr2=0x7fdcf019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.233+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf01042b0 0x7fdcf019a7e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdce000b450 tx=0x7fdce000b910 comp rx=0 tx=0).stop 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 shutdown_connections 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:6800/2425110073,v1:172.21.15.19:6801/2425110073] conn(0x7fdcd80afce0 0x7fdcd80b21f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcf0109fb0 0x7fdcf019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcf0109b10 0x7fdcf019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 --2- 172.21.15.19:0/1273091577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcf01042b0 0x7fdcf019a7e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 >> 172.21.15.19:0/1273091577 conn(0x7fdcf00ff360 msgr2=0x7fdcf010f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:05:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 shutdown_connections 2024-04-25T11:05:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:05:27.234+0000 7fdcf71a8640 1 -- 172.21.15.19:0/1273091577 wait complete. 2024-04-25T11:05:27.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:05:27.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:05:27.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:05:27.341 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T11:05:27.342 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:05:27.358 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:05:27.376 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:05:27.398 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T11:05:27.398 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T11:05:27.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a 2024-04-25T11:05:27.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:27 smithi169 ceph-mon[33049]: pgmap v340: 97 pgs: 97 active+clean; 11 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 307 B/s wr, 2 op/s 2024-04-25T11:05:27.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:27 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/3122897813' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:05:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:27 smithi169 ceph-mon[33049]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T11:05:27.501 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:27 smithi169 ceph-mon[33049]: from='client.? 172.21.15.19:0/1273091577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:05:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:27 smithi112 ceph-mon[32799]: pgmap v340: 97 pgs: 97 active+clean; 11 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 307 B/s wr, 2 op/s 2024-04-25T11:05:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:27 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/3122897813' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:05:27.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:27 smithi112 ceph-mon[32799]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T11:05:27.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:27 smithi112 ceph-mon[32799]: from='client.? 172.21.15.19:0/1273091577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:05:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:27 smithi019 ceph-mon[29209]: pgmap v340: 97 pgs: 97 active+clean; 11 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 307 B/s wr, 2 op/s 2024-04-25T11:05:27.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:27 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/3122897813' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:05:27.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:27 smithi019 ceph-mon[29209]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T11:05:27.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:27 smithi019 ceph-mon[29209]: from='client.? 172.21.15.19:0/1273091577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:05:27.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:27 smithi019 systemd[1]: Stopping Ceph mon.a for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:28.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a[29205]: 2024-04-25T11:05:27.845+0000 7f39ad6f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:28.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a[29205]: 2024-04-25T11:05:27.845+0000 7f39ad6f7700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T11:05:28.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:28 smithi019 podman[94016]: 2024-04-25 11:05:28.020905406 +0000 UTC m=+0.424853508 container died ff48db461b118cb31cf002a543c99c86ae88d5be590aaf735aea7a09e3bed8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD) 2024-04-25T11:05:28.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:28 smithi019 podman[94016]: 2024-04-25 11:05:28.38388198 +0000 UTC m=+0.787830015 container cleanup ff48db461b118cb31cf002a543c99c86ae88d5be590aaf735aea7a09e3bed8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T11:05:28.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:05:28 smithi019 bash[94016]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-a 2024-04-25T11:05:28.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T11:05:28.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:28.933 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T11:05:28.934 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T11:05:28.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.b 2024-04-25T11:05:29.253 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:28 smithi112 systemd[1]: Stopping Ceph mon.b for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:29.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-b[32795]: 2024-04-25T11:05:29.252+0000 7f79448e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:29.558 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-b[32795]: 2024-04-25T11:05:29.252+0000 7f79448e6700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T11:05:29.559 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:29 smithi112 podman[67864]: 2024-04-25 11:05:29.416094155 +0000 UTC m=+0.288664253 container died 2062cf52b06d50f566f8b499a1dc61db9a7f9a22de819a7bf5c7db0eaadf8b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T11:05:30.058 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:29 smithi112 podman[67864]: 2024-04-25 11:05:29.769352834 +0000 UTC m=+0.641922930 container cleanup 2062cf52b06d50f566f8b499a1dc61db9a7f9a22de819a7bf5c7db0eaadf8b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:05:30.059 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 11:05:29 smithi112 bash[67864]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-b 2024-04-25T11:05:30.187 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T11:05:30.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:30.229 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T11:05:30.229 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T11:05:30.229 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.c 2024-04-25T11:05:30.546 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:30 smithi169 systemd[1]: Stopping Ceph mon.c for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:30.838 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c[33045]: 2024-04-25T11:05:30.545+0000 7ff69aee5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:30.839 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c[33045]: 2024-04-25T11:05:30.545+0000 7ff69aee5700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T11:05:30.839 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:30 smithi169 podman[69237]: 2024-04-25 11:05:30.56637051 +0000 UTC m=+0.141781956 container died 9313eeb8a1be24f016b0ef40156fef5e3de7d66b0d628053c011a8c4f63cffb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T11:05:31.180 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:30 smithi169 podman[69237]: 2024-04-25 11:05:30.945975857 +0000 UTC m=+0.521387292 container cleanup 9313eeb8a1be24f016b0ef40156fef5e3de7d66b0d628053c011a8c4f63cffb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T11:05:31.180 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:05:30 smithi169 bash[69237]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mon-c 2024-04-25T11:05:31.300 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T11:05:31.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:31.350 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T11:05:31.351 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T11:05:31.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.x 2024-04-25T11:05:31.688 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:05:31 smithi019 systemd[1]: Stopping Ceph mgr.x for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:32.014 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:05:31 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:05:31.687+0000 7f48b2ed8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:32.015 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:05:31 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:05:31.687+0000 7f48b2ed8700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T11:05:32.015 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:05:31 smithi019 podman[94136]: 2024-04-25 11:05:31.719189495 +0000 UTC m=+0.161990311 container died 51e0f592b2ee4285ee39f29f1fbb208ae88e0451acea302f3019dabb5d44afff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T11:05:32.349 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:05:32 smithi019 podman[94136]: 2024-04-25 11:05:32.130330703 +0000 UTC m=+0.573131518 container cleanup 51e0f592b2ee4285ee39f29f1fbb208ae88e0451acea302f3019dabb5d44afff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T11:05:32.349 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:05:32 smithi019 bash[94136]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-x 2024-04-25T11:05:32.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T11:05:32.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:32.523 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T11:05:32.523 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T11:05:32.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.y 2024-04-25T11:05:32.808 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 11:05:32 smithi112 systemd[1]: Stopping Ceph mgr.y for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:33.168 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 11:05:32 smithi112 podman[67987]: 2024-04-25 11:05:32.873166034 +0000 UTC m=+0.160249453 container died 9d341b766b7d6a3fcc4f075ae315d73cea4e9dd22d58cfafebb482b680abf10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:05:33.493 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 11:05:33 smithi112 podman[67987]: 2024-04-25 11:05:33.267607463 +0000 UTC m=+0.554690889 container cleanup 9d341b766b7d6a3fcc4f075ae315d73cea4e9dd22d58cfafebb482b680abf10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T11:05:33.493 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 11:05:33 smithi112 bash[67987]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-y 2024-04-25T11:05:33.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T11:05:33.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:33.636 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T11:05:33.637 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T11:05:33.637 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.z 2024-04-25T11:05:33.997 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:05:33 smithi169 systemd[1]: Stopping Ceph mgr.z for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:34.250 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:05:33 smithi169 podman[69360]: 2024-04-25 11:05:33.997948359 +0000 UTC m=+0.162817402 container died 43f2139ac85b33cad3f7f381c8489809bba13ec3ae7c5a1f578ca9baed59d89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T11:05:34.634 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:05:34 smithi169 podman[69360]: 2024-04-25 11:05:34.358781729 +0000 UTC m=+0.523650781 container cleanup 43f2139ac85b33cad3f7f381c8489809bba13ec3ae7c5a1f578ca9baed59d89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T11:05:34.635 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:05:34 smithi169 bash[69360]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mgr-z 2024-04-25T11:05:34.780 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T11:05:34.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:34.813 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T11:05:34.813 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T11:05:34.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.0 2024-04-25T11:05:35.110 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:34 smithi019 systemd[1]: Stopping Ceph osd.0 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:35.610 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0[43995]: 2024-04-25T11:05:35.145+0000 7fcaf609a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:35.611 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0[43995]: 2024-04-25T11:05:35.145+0000 7fcaf609a700 -1 osd.0 91 *** Got signal Terminated *** 2024-04-25T11:05:35.611 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0[43995]: 2024-04-25T11:05:35.145+0000 7fcaf609a700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:05:40.761 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:40 smithi019 podman[94260]: 2024-04-25 11:05:40.483183645 +0000 UTC m=+5.465150252 container died a13218be2fca2ddfabc9dd2aef46d0827c083d66b3fb7400cf6b853a12c5e4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T11:05:41.110 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:40 smithi019 podman[94260]: 2024-04-25 11:05:40.8515836 +0000 UTC m=+5.833550206 container cleanup a13218be2fca2ddfabc9dd2aef46d0827c083d66b3fb7400cf6b853a12c5e4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T11:05:41.110 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:40 smithi019 bash[94260]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0 2024-04-25T11:05:41.478 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:41 smithi019 podman[94278]: 2024-04-25 11:05:41.111205794 +0000 UTC m=+0.619454027 container remove a13218be2fca2ddfabc9dd2aef46d0827c083d66b3fb7400cf6b853a12c5e4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T11:05:41.772 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:41 smithi019 podman[94366]: 2024-04-25 11:05:41.381501774 +0000 UTC m=+0.018710388 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:05:41.772 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:41 smithi019 podman[94366]: 2024-04-25 11:05:41.500778384 +0000 UTC m=+0.137987029 container create 8fd5be04aac53f0a59b0d196af661e6b55349dc8554875e25189fc4e6c4e1df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:05:41.773 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:41 smithi019 podman[94366]: 2024-04-25 11:05:41.77327837 +0000 UTC m=+0.410486970 container init 8fd5be04aac53f0a59b0d196af661e6b55349dc8554875e25189fc4e6c4e1df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T11:05:42.097 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:41 smithi019 podman[94366]: 2024-04-25 11:05:41.776679845 +0000 UTC m=+0.413888452 container start 8fd5be04aac53f0a59b0d196af661e6b55349dc8554875e25189fc4e6c4e1df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T11:05:42.097 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:41 smithi019 podman[94366]: 2024-04-25 11:05:41.88156652 +0000 UTC m=+0.518775133 container attach 8fd5be04aac53f0a59b0d196af661e6b55349dc8554875e25189fc4e6c4e1df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:05:42.360 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:05:42 smithi019 podman[94366]: 2024-04-25 11:05:42.097615763 +0000 UTC m=+0.734824372 container died 8fd5be04aac53f0a59b0d196af661e6b55349dc8554875e25189fc4e6c4e1df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T11:05:42.662 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T11:05:42.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:42.703 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T11:05:42.703 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T11:05:42.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.1 2024-04-25T11:05:42.997 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:42 smithi112 systemd[1]: Stopping Ceph osd.1 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:43.308 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1[37412]: 2024-04-25T11:05:42.996+0000 7f2c79ab2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:43.308 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1[37412]: 2024-04-25T11:05:42.996+0000 7f2c79ab2700 -1 osd.1 91 *** Got signal Terminated *** 2024-04-25T11:05:43.308 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1[37412]: 2024-04-25T11:05:42.996+0000 7f2c79ab2700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:05:48.558 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:48 smithi112 podman[68122]: 2024-04-25 11:05:48.305889207 +0000 UTC m=+5.418528072 container died 5a0315afdfc7c8bd26b7324fb7adc24c719db0259487598a7993f6fa6acbf49a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T11:05:48.989 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:48 smithi112 podman[68122]: 2024-04-25 11:05:48.646997811 +0000 UTC m=+5.759636684 container cleanup 5a0315afdfc7c8bd26b7324fb7adc24c719db0259487598a7993f6fa6acbf49a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:05:48.990 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:48 smithi112 bash[68122]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1 2024-04-25T11:05:49.308 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:48 smithi112 podman[68142]: 2024-04-25 11:05:48.990026699 +0000 UTC m=+0.679149139 container remove 5a0315afdfc7c8bd26b7324fb7adc24c719db0259487598a7993f6fa6acbf49a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T11:05:49.726 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:49 smithi112 podman[68233]: 2024-04-25 11:05:49.316540229 +0000 UTC m=+0.018208757 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:05:49.726 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:49 smithi112 podman[68233]: 2024-04-25 11:05:49.441145797 +0000 UTC m=+0.142814300 container create e7699e4c3fb653fc29ee46efc1ebe17efee18e759aa589280b7799361606a6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:05:49.727 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:49 smithi112 podman[68233]: 2024-04-25 11:05:49.64351053 +0000 UTC m=+0.345179033 container init e7699e4c3fb653fc29ee46efc1ebe17efee18e759aa589280b7799361606a6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T11:05:49.727 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:49 smithi112 podman[68233]: 2024-04-25 11:05:49.646662666 +0000 UTC m=+0.348331172 container start e7699e4c3fb653fc29ee46efc1ebe17efee18e759aa589280b7799361606a6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T11:05:50.058 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:49 smithi112 podman[68233]: 2024-04-25 11:05:49.726690488 +0000 UTC m=+0.428359005 container attach e7699e4c3fb653fc29ee46efc1ebe17efee18e759aa589280b7799361606a6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T11:05:50.059 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:49 smithi112 podman[68233]: 2024-04-25 11:05:49.940394536 +0000 UTC m=+0.642063065 container died e7699e4c3fb653fc29ee46efc1ebe17efee18e759aa589280b7799361606a6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T11:05:50.574 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 11:05:50 smithi112 podman[68259]: 2024-04-25 11:05:50.556539165 +0000 UTC m=+0.610878108 container remove e7699e4c3fb653fc29ee46efc1ebe17efee18e759aa589280b7799361606a6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T11:05:50.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T11:05:50.622 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:50.622 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T11:05:50.622 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T11:05:50.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.2 2024-04-25T11:05:50.946 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:50 smithi169 systemd[1]: Stopping Ceph osd.2 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:51.250 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2[38210]: 2024-04-25T11:05:50.944+0000 7f401e4db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:51.250 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2[38210]: 2024-04-25T11:05:50.944+0000 7f401e4db700 -1 osd.2 91 *** Got signal Terminated *** 2024-04-25T11:05:51.250 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2[38210]: 2024-04-25T11:05:50.944+0000 7f401e4db700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:05:56.537 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:56 smithi169 podman[69498]: 2024-04-25 11:05:56.278080032 +0000 UTC m=+5.461288669 container died 01686abd820b02f22ace99f1a51bc4a1741e1dfb454115f58508455a2a58e41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T11:05:56.929 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:56 smithi169 podman[69498]: 2024-04-25 11:05:56.637300539 +0000 UTC m=+5.820509187 container cleanup 01686abd820b02f22ace99f1a51bc4a1741e1dfb454115f58508455a2a58e41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:05:56.929 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:56 smithi169 bash[69498]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2 2024-04-25T11:05:57.250 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:56 smithi169 podman[69515]: 2024-04-25 11:05:56.930127566 +0000 UTC m=+0.653000614 container remove 01686abd820b02f22ace99f1a51bc4a1741e1dfb454115f58508455a2a58e41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:05:57.516 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:57 smithi169 podman[69610]: 2024-04-25 11:05:57.162473125 +0000 UTC m=+0.018350092 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:05:57.516 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:57 smithi169 podman[69610]: 2024-04-25 11:05:57.264425488 +0000 UTC m=+0.120302497 container create 5a3a685f113dcd0f0a77ec9bd4fd59e209ebe2beb99ff2cb9c7aca39fa805685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T11:05:57.809 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:57 smithi169 podman[69610]: 2024-04-25 11:05:57.517304014 +0000 UTC m=+0.373180983 container init 5a3a685f113dcd0f0a77ec9bd4fd59e209ebe2beb99ff2cb9c7aca39fa805685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-25T11:05:57.809 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:57 smithi169 podman[69610]: 2024-04-25 11:05:57.520537873 +0000 UTC m=+0.376414840 container start 5a3a685f113dcd0f0a77ec9bd4fd59e209ebe2beb99ff2cb9c7aca39fa805685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:05:57.809 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:57 smithi169 podman[69610]: 2024-04-25 11:05:57.608882133 +0000 UTC m=+0.464759115 container attach 5a3a685f113dcd0f0a77ec9bd4fd59e209ebe2beb99ff2cb9c7aca39fa805685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T11:05:58.161 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 11:05:57 smithi169 podman[69610]: 2024-04-25 11:05:57.809893177 +0000 UTC m=+0.665770152 container died 5a3a685f113dcd0f0a77ec9bd4fd59e209ebe2beb99ff2cb9c7aca39fa805685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T11:05:58.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T11:05:58.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:05:58.519 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T11:05:58.519 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T11:05:58.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.3 2024-04-25T11:05:58.837 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:05:58 smithi019 systemd[1]: Stopping Ceph osd.3 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:05:59.110 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:05:58 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T11:05:58.836+0000 7f1cf3536700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:05:59.111 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:05:58 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T11:05:58.836+0000 7f1cf3536700 -1 osd.3 91 *** Got signal Terminated *** 2024-04-25T11:05:59.111 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:05:58 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T11:05:58.836+0000 7f1cf3536700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:01.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:01.462+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:02.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:01 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:01.662+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:02.110 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:01 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T11:06:01.727+0000 7f1cebae4700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.463636+0000 front 2024-04-25T11:05:37.463706+0000 (oldest deadline 2024-04-25T11:06:00.864116+0000) 2024-04-25T11:06:02.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:01 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:01.699+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:02.741 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:02 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:02.675+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:02.741 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:02 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:02.489+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:02.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:02.476+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:03.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:02 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:02.649+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:03.110 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:02 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T11:06:02.740+0000 7f1cebae4700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.463636+0000 front 2024-04-25T11:05:37.463706+0000 (oldest deadline 2024-04-25T11:06:00.864116+0000) 2024-04-25T11:06:03.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:03 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:03.525+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:03.861 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:03 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3[50918]: 2024-04-25T11:06:03.747+0000 7f1cebae4700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.463636+0000 front 2024-04-25T11:05:37.463706+0000 (oldest deadline 2024-04-25T11:06:00.864116+0000) 2024-04-25T11:06:03.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:03 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:03.700+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:04.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:03.504+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:04.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:03 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:03.633+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:04.499 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:04 smithi019 podman[94474]: 2024-04-25 11:06:04.232433802 +0000 UTC m=+5.502765701 container died c4f5a22e9354b69e03aeb13f95306c6eb78c79af7eb459fea0a828ab7b4dc5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T11:06:04.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:04.470+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:04.796 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:04 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:04.500+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:04.797 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:04 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:04.692+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:04.797 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:04 smithi019 podman[94474]: 2024-04-25 11:06:04.587471617 +0000 UTC m=+5.857803512 container cleanup c4f5a22e9354b69e03aeb13f95306c6eb78c79af7eb459fea0a828ab7b4dc5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T11:06:04.797 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:04 smithi019 bash[94474]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3 2024-04-25T11:06:05.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:04 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:04.591+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:05.110 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:04 smithi019 podman[94505]: 2024-04-25 11:06:04.796891229 +0000 UTC m=+0.555599900 container remove c4f5a22e9354b69e03aeb13f95306c6eb78c79af7eb459fea0a828ab7b4dc5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T11:06:05.471 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:05 smithi019 podman[94598]: 2024-04-25 11:06:05.123109292 +0000 UTC m=+0.018471272 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:06:05.471 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:05 smithi019 podman[94598]: 2024-04-25 11:06:05.239411118 +0000 UTC m=+0.134773112 container create b8a2c1a9318c1184629ec8b903924d0e7c9d85f0d5cc0be597a3cfce606465df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:06:05.471 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:05 smithi019 podman[94598]: 2024-04-25 11:06:05.458530448 +0000 UTC m=+0.353892411 container init b8a2c1a9318c1184629ec8b903924d0e7c9d85f0d5cc0be597a3cfce606465df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T11:06:05.471 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:05 smithi019 podman[94598]: 2024-04-25 11:06:05.461914345 +0000 UTC m=+0.357276301 container start b8a2c1a9318c1184629ec8b903924d0e7c9d85f0d5cc0be597a3cfce606465df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-25T11:06:05.728 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:05 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:05.470+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:05.729 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:05 smithi019 podman[94598]: 2024-04-25 11:06:05.575140122 +0000 UTC m=+0.470502087 container attach b8a2c1a9318c1184629ec8b903924d0e7c9d85f0d5cc0be597a3cfce606465df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:06:05.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:05.493+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:05.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:05.443+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:06.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:05 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:05.599+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:06.060 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:05 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:05.728+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:06.061 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 11:06:05 smithi019 podman[94598]: 2024-04-25 11:06:05.769430626 +0000 UTC m=+0.664792597 container died b8a2c1a9318c1184629ec8b903924d0e7c9d85f0d5cc0be597a3cfce606465df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:06:06.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:05.866+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:06.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T11:06:06.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:06:06.427 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T11:06:06.427 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T11:06:06.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.4 2024-04-25T11:06:06.610 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:06.508+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:06.726 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:06 smithi112 systemd[1]: Stopping Ceph osd.4 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:06:06.727 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:06 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:06.595+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:06.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:06.426+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:06.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:06.513+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:07.058 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:06 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4[44184]: 2024-04-25T11:06:06.725+0000 7fb43829d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:06:07.058 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:06 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4[44184]: 2024-04-25T11:06:06.725+0000 7fb43829d700 -1 osd.4 91 *** Got signal Terminated *** 2024-04-25T11:06:07.058 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:06 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4[44184]: 2024-04-25T11:06:06.725+0000 7fb43829d700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:07.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:06 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:06.695+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:07.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:06.907+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:07.385+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:07.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:07.528+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:07.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:07 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:07.520+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:07.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:07 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:07.671+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:08.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:07 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:07.587+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:08.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:07.939+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:08.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:08 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:08.573+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:08.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:08 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:08.339+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:08.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:08 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:08.484+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:08.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:08 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:08.717+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:09.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:08 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:08.593+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:09.174 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:08 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:08.889+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:09.174 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:08 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:08.889+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:09.500 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:09 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:09.385+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:09.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:09 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:09.385+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:09.726 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:09 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:09.465+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:09.726 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:09 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:09.465+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:09.897 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:09 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:09.530+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:10.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:09 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:09.640+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:10.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:09 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:09.640+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:10.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:09 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:09.725+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:10.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:09 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:09.897+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:10.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:09 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:09.897+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:10.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:10 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:10.483+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:10.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:10 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:10.387+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:10.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:10 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:10.387+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:10.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:10 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:10.682+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:10.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:10 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:10.682+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:10.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:10 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:10.509+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:10.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:10 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:10.509+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:11.058 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:10 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4[44184]: 2024-04-25T11:06:10.730+0000 7fb43084b700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:45.695412+0000 front 2024-04-25T11:05:45.695434+0000 (oldest deadline 2024-04-25T11:06:09.795556+0000) 2024-04-25T11:06:11.059 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:10 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:10.645+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:11.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:10 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:10.648+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:11.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:10 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:10.648+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:11.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:10 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:10.899+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:11.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:10 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:10.899+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:11.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:11 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:11.436+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:11.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:11 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:11.411+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:11.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:11 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:11.411+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:11.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:11 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:11.499+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:11.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:11 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:11.499+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:11.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:11 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:11.703+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:11.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:11 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:11.703+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:11.960 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:11 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4[44184]: 2024-04-25T11:06:11.694+0000 7fb43084b700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:45.695412+0000 front 2024-04-25T11:05:45.695434+0000 (oldest deadline 2024-04-25T11:06:09.795556+0000) 2024-04-25T11:06:11.960 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:11 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:11.638+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:11.960 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:11 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:11.610+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:11.960 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:11 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:11.610+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:12.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:11 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:11.901+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:12.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:11 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:11.901+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:12.308 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:12 smithi112 podman[68341]: 2024-04-25 11:06:12.072818661 +0000 UTC m=+5.455079101 container died 6fc74b49eaa123d7dd7fa624d535a08dac681e1aaa3046fcdfdccd899a2beeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T11:06:12.632 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:12 smithi112 podman[68341]: 2024-04-25 11:06:12.451466889 +0000 UTC m=+5.833727334 container cleanup 6fc74b49eaa123d7dd7fa624d535a08dac681e1aaa3046fcdfdccd899a2beeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T11:06:12.632 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:12 smithi112 bash[68341]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4 2024-04-25T11:06:12.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:12 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:12.481+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:12.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:12 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:12.377+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:12.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:12 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:12.377+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:12.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:12 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:12.530+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:12.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:12 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:12.530+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:12.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:12 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:12.728+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:12.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:12 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:12.728+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:13.003 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:12 smithi112 podman[68358]: 2024-04-25 11:06:12.677956445 +0000 UTC m=+0.600212105 container remove 6fc74b49eaa123d7dd7fa624d535a08dac681e1aaa3046fcdfdccd899a2beeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-25T11:06:13.003 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:12 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:12.632+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:13.003 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:12 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:12.644+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:13.003 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:12 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:12.644+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:13.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:12 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:12.879+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:13.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:12 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:12.879+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:13.308 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:13 smithi112 podman[68451]: 2024-04-25 11:06:12.906131033 +0000 UTC m=+0.017855880 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:06:13.309 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:13 smithi112 podman[68451]: 2024-04-25 11:06:13.012702938 +0000 UTC m=+0.124427752 container create e3b58ffa2c3ac6e474fc138f2e59a22750737d9fc91ea5f097a342d35bddd29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5) 2024-04-25T11:06:13.309 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:13 smithi112 podman[68451]: 2024-04-25 11:06:13.240195787 +0000 UTC m=+0.351920600 container init e3b58ffa2c3ac6e474fc138f2e59a22750737d9fc91ea5f097a342d35bddd29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:06:13.309 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:13 smithi112 podman[68451]: 2024-04-25 11:06:13.243292379 +0000 UTC m=+0.355017197 container start e3b58ffa2c3ac6e474fc138f2e59a22750737d9fc91ea5f097a342d35bddd29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T11:06:13.606 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:13 smithi112 podman[68451]: 2024-04-25 11:06:13.331619577 +0000 UTC m=+0.443344391 container attach e3b58ffa2c3ac6e474fc138f2e59a22750737d9fc91ea5f097a342d35bddd29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T11:06:13.606 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 11:06:13 smithi112 podman[68451]: 2024-04-25 11:06:13.536324568 +0000 UTC m=+0.648049393 container died e3b58ffa2c3ac6e474fc138f2e59a22750737d9fc91ea5f097a342d35bddd29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T11:06:13.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:13 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:13.522+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:13.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:13 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:13.398+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:13.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:13 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:13.398+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:13.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:13 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:13.518+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:13.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:13 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:13.518+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:13.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:13 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:13.711+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:13.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:13 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:13.711+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:14.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:13 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:13.606+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:14.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:13 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:13.645+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:14.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:13 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:13.645+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:14.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T11:06:14.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:06:14.205 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T11:06:14.205 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T11:06:14.205 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.5 2024-04-25T11:06:14.231 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:13 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:13.837+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:14.231 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:13 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:13.837+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:14.484 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:14 smithi169 systemd[1]: Stopping Ceph osd.5 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:06:14.486 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:14.447+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:14.486 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:14.447+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:14.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:14.484+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:14.751 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:14.515+0000 7fe961bcf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:06:14.751 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:14.515+0000 7fe961bcf700 -1 osd.5 91 *** Got signal Terminated *** 2024-04-25T11:06:14.751 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:14.515+0000 7fe961bcf700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:14.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:14 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:14.555+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:14.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:14 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:14.555+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:14.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:14 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:14.729+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:14.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:14 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:14.729+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:15.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:14 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:14.589+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:15.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:14 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:14.690+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:15.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:14 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:14.690+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:15.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:14.787+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:15.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:14 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:14.787+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:15.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:15 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:15.439+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:15.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:15 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:15.454+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:15.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:15 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:15.454+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:15.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:15 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:15.776+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:15.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:15 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:15.776+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:15.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:15 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:15.584+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:15.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:15 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:15.584+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:16.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:15 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:15.583+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:16.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:15 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:15.709+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:16.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:15 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:15.709+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:16.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:15 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:15.818+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:16.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:15 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:15.818+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:16.742 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:16 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:16.556+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:16.743 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:16 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:16.556+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:16.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:16 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:16.392+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:16.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:16 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:16.475+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:16.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:16 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:16.475+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:16.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:16 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:16.535+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:16.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:16 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:16.720+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:16.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:16 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:16.720+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:17.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:16 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:16.742+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:17.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:16 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:16.742+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:17.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:16 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:16.813+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:17.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:16 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:16.813+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:17.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:17 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:17.347+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:17.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:17 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:17.450+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:17.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:17 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:17.450+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:17.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:17 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:17.728+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:17.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:17 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:17.728+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:17.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:17 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:17.573+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:17.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:17 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:17.573+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:18.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:17 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:17.564+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:18.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:17 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:17.758+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:18.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:17 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:17.758+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:18.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:17 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:17.839+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:18.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:17 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:17.839+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:18.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:17 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:17.839+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:52.501203+0000 front 2024-04-25T11:05:52.501263+0000 (oldest deadline 2024-04-25T11:06:17.801391+0000) 2024-04-25T11:06:18.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:18.340+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:18.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:18.340+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:18.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:18.485+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:18.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:18.485+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:18.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:18.485+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:18.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:18 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:18.576+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:18.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:18 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:18.576+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:18.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:18 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:18.706+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:18.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:18 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:18.706+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:19.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:18 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:18.578+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:19.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:18 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:18.578+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:19.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:18 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:18.723+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:19.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:18 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:18.723+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:19.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:18 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:18.723+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:19.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:18.811+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:37.998382+0000 front 2024-04-25T11:05:37.998506+0000 (oldest deadline 2024-04-25T11:06:04.999055+0000) 2024-04-25T11:06:19.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:18.811+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.999864+0000 front 2024-04-25T11:05:44.999822+0000 (oldest deadline 2024-04-25T11:06:08.499630+0000) 2024-04-25T11:06:19.251 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:18 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5[45003]: 2024-04-25T11:06:18.811+0000 7fe95e185700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:52.501203+0000 front 2024-04-25T11:05:52.501263+0000 (oldest deadline 2024-04-25T11:06:17.801391+0000) 2024-04-25T11:06:19.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:19 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:19.312+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:19.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:19 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:19.312+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:19.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:19 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:19.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:19.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:19 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:19.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:19.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:19 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:19.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:19.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:19 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:19.689+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:19.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:19 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:19.689+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:19.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:19 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:19.689+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:19.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:19 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:19.557+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:19.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:19 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:19.557+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:20.110 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:19 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:19.626+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:20.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:19 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:19.626+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:20.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:19 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:19.626+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:20.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:19 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:19.710+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:20.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:19 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:19.710+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:20.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:19 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:19.710+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:20.216 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:19 smithi169 podman[69728]: 2024-04-25 11:06:19.930954911 +0000 UTC m=+5.527639295 container died 97361c3f477333b4cc953b7abeee8797d79af7bdcbe3527823120da585ecfadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T11:06:20.501 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:20 smithi169 podman[69728]: 2024-04-25 11:06:20.314812915 +0000 UTC m=+5.911497309 container cleanup 97361c3f477333b4cc953b7abeee8797d79af7bdcbe3527823120da585ecfadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T11:06:20.501 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:20 smithi169 bash[69728]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5 2024-04-25T11:06:20.501 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:20 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:20.305+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:20.501 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:20 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:20.305+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:20.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:20 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:20.432+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:20.502 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:20 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:20.432+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:20.502 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:20 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:20.432+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:20.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:20.667+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:20.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:20.667+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:20.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:20.667+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:20.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:20.589+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:20.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:20.589+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:20.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:20 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:20.589+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:20.934 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:20 smithi169 podman[69746]: 2024-04-25 11:06:20.599653902 +0000 UTC m=+0.659343213 container remove 97361c3f477333b4cc953b7abeee8797d79af7bdcbe3527823120da585ecfadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T11:06:21.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:20 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:20.576+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:21.059 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:20 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:20.576+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:21.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:20 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:20.682+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:21.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:20 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:20.682+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:21.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:20 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:20.682+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:21.244 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:20 smithi169 podman[69840]: 2024-04-25 11:06:20.838362942 +0000 UTC m=+0.018475739 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:06:21.244 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:20 smithi169 podman[69840]: 2024-04-25 11:06:20.942014327 +0000 UTC m=+0.122127107 container create 70874c9160f315d6b3b9b0130110b05c5adc1f63e5de7f3e22445d50c537d811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T11:06:21.244 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:21 smithi169 podman[69840]: 2024-04-25 11:06:21.152806872 +0000 UTC m=+0.332919656 container init 70874c9160f315d6b3b9b0130110b05c5adc1f63e5de7f3e22445d50c537d811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T11:06:21.244 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:21 smithi169 podman[69840]: 2024-04-25 11:06:21.155804177 +0000 UTC m=+0.335916959 container start 70874c9160f315d6b3b9b0130110b05c5adc1f63e5de7f3e22445d50c537d811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T11:06:21.501 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:21 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:21.344+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:21.501 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:21 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:21.344+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:21.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:21 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:21.403+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:21.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:21 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:21.403+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:21.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:21 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:21.403+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:21.502 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:21 smithi169 podman[69840]: 2024-04-25 11:06:21.2444635 +0000 UTC m=+0.424576282 container attach 70874c9160f315d6b3b9b0130110b05c5adc1f63e5de7f3e22445d50c537d811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T11:06:21.502 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 11:06:21 smithi169 podman[69840]: 2024-04-25 11:06:21.452876166 +0000 UTC m=+0.632988950 container died 70874c9160f315d6b3b9b0130110b05c5adc1f63e5de7f3e22445d50c537d811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T11:06:21.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:21 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:21.598+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:21.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:21 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:21.598+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:21.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:21 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:21.598+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:21.862 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:21 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:21.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:21.862 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:21 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:21.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:21.862 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:21 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:21.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:22.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:21 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:21.597+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:22.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:21 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:21.597+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:22.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:21 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:21.668+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:22.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:21 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:21.668+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:22.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:21 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:21.668+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:22.195 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T11:06:22.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:06:22.280 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T11:06:22.280 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T11:06:22.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.6 2024-04-25T11:06:22.500 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:22.390+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:22.501 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:22.390+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:22.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:22.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:22.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:22.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:22.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:22 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:22.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:22.588 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:22 smithi019 systemd[1]: Stopping Ceph osd.6 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:06:22.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:22.722+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:22.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:22.722+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:22.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:22.722+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:22.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:22.587+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:22.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:22.587+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:22.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:22.587+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:22.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:22.594+0000 7fa530d83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:06:22.862 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:22.595+0000 7fa530d83700 -1 osd.6 91 *** Got signal Terminated *** 2024-04-25T11:06:22.862 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:22.595+0000 7fa530d83700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:23.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:22.579+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:23.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:22.579+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:23.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:22.625+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:23.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:22.625+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:23.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:22 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:22.625+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:23.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:23.433+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:23.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:23.433+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:23.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:23.377+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:23.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:23.377+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:23.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:23 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:23.377+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:24.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:23.589+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:24.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:23.589+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:24.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:23.587+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:24.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:23.587+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:24.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:23 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:23.587+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:24.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:23.615+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:24.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:23.615+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:24.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:23.615+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:24.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:23.672+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:24.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:23.672+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:24.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:23 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:23.672+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:24.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:24.405+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:24.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:24.405+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:24.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:24.405+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:24.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:24.408+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:24.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:24.408+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:24.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:24 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:24.408+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:24.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:24.546+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:24.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:24.546+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:24.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:24.546+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:24.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:24.546+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:24.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:24 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:24.546+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:24.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:24.603+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:24.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:24.603+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:24.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:24.603+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:24.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:24.623+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:24.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:24.623+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:24.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:24 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:24.623+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:25.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:25.413+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:25.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:25.413+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:25.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:25.413+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:25.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:25.416+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:25.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:25.416+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:25.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:25 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:25.416+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:25.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:25.609+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:25.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:25.609+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:25.861 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:25.609+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:25.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:25.634+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:25.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:25.634+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:25.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:25.634+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:25.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:25 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:25.634+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:26.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:25.590+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:26.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:25.590+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:26.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:25.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:26.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:25.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:26.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:25 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:25.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:26.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:26.390+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:26.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:26.390+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:26.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:26.390+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:26.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:26.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:26.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:26.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:26.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:26 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:26.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:26.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:26.556+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:26.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:26.556+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:26.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:26.536+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:26.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:26.536+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:26.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:26 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:26.536+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:26.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:26.594+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:26.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:26.594+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:26.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:26.594+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:26.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:26.660+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:26.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:26.660+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:26.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:26.660+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:26.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:26 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:26.660+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:27.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:27.403+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:27.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:27.403+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:27.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:27.404+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:27.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:27.404+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:27.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:27.404+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:27.891 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:27 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:27.405+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:27.893 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:27.563+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.438733+0000 front 2024-04-25T11:05:39.438864+0000 (oldest deadline 2024-04-25T11:06:02.338951+0000) 2024-04-25T11:06:27.893 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:27.563+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.640412+0000 front 2024-04-25T11:05:44.640308+0000 (oldest deadline 2024-04-25T11:06:09.340335+0000) 2024-04-25T11:06:27.893 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:27.563+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.441435+0000 front 2024-04-25T11:05:53.441506+0000 (oldest deadline 2024-04-25T11:06:19.341552+0000) 2024-04-25T11:06:27.893 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6[57641]: 2024-04-25T11:06:27.563+0000 7fa529331700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.442326+0000 front 2024-04-25T11:06:03.442260+0000 (oldest deadline 2024-04-25T11:06:26.942460+0000) 2024-04-25T11:06:27.893 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:27.586+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:27.894 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:27.586+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:27.894 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:27.586+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:27.894 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:27.706+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:27.894 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:27.706+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:27.894 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:27.706+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:27.894 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:27 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:27.706+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:27.894 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:27.487+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:27.895 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:27.487+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:27.895 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:27.487+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:28.238 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:27 smithi019 podman[94703]: 2024-04-25 11:06:27.964425579 +0000 UTC m=+5.478974458 container died a030e9c52e3b17f2901abd32fdff4f8ac1a4668b47e7bf3abd7294daa410fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T11:06:28.596 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:28 smithi019 podman[94703]: 2024-04-25 11:06:28.321168956 +0000 UTC m=+5.835717843 container cleanup a030e9c52e3b17f2901abd32fdff4f8ac1a4668b47e7bf3abd7294daa410fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T11:06:28.596 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:28 smithi019 bash[94703]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6 2024-04-25T11:06:28.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:28.382+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:28.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:28.382+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:28.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:28.382+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:28.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:28.452+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:28.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:28.452+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:28.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:28 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:28.452+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:28.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:28.551+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:28.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:28.551+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:28.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:28.551+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:28.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:28.518+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:28.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:28.518+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:28.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:28.518+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:28.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:28 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:28.518+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:28.860 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:28 smithi019 podman[94721]: 2024-04-25 11:06:28.597065255 +0000 UTC m=+0.625152484 container remove a030e9c52e3b17f2901abd32fdff4f8ac1a4668b47e7bf3abd7294daa410fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T11:06:28.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:28.739+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:28.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:28.739+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:28.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:28.739+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:28.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:28 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:28.739+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:29.258 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:28 smithi019 podman[94810]: 2024-04-25 11:06:28.854736131 +0000 UTC m=+0.018380977 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:06:29.258 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:28 smithi019 podman[94810]: 2024-04-25 11:06:28.95660739 +0000 UTC m=+0.120252277 container create 87f43254096565f557f743bf74ddce573fa1e998103487ff33d94199b9fdbc4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T11:06:29.258 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:29 smithi019 podman[94810]: 2024-04-25 11:06:29.183793554 +0000 UTC m=+0.347438375 container init 87f43254096565f557f743bf74ddce573fa1e998103487ff33d94199b9fdbc4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T11:06:29.258 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:29 smithi019 podman[94810]: 2024-04-25 11:06:29.187146095 +0000 UTC m=+0.350790927 container start 87f43254096565f557f743bf74ddce573fa1e998103487ff33d94199b9fdbc4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T11:06:29.259 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:29 smithi019 podman[94810]: 2024-04-25 11:06:29.258734077 +0000 UTC m=+0.422378910 container attach 87f43254096565f557f743bf74ddce573fa1e998103487ff33d94199b9fdbc4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-25T11:06:29.614 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 11:06:29 smithi019 podman[94810]: 2024-04-25 11:06:29.492813892 +0000 UTC m=+0.656458705 container died 87f43254096565f557f743bf74ddce573fa1e998103487ff33d94199b9fdbc4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T11:06:29.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:29.427+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:29.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:29.427+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:29.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:29.427+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:29.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:29.450+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:29.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:29.450+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:29.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:29 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:29.450+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:29.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:29.520+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:29.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:29.520+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:29.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:29.520+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:29.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:29.500+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:29.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:29.500+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:29.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:29.500+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:29.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:29 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:29.500+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:30.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:29.752+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:30.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:29.752+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:30.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:29.752+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:30.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:29 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:29.752+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:30.239 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T11:06:30.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:06:30.285 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T11:06:30.285 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T11:06:30.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.7 2024-04-25T11:06:30.597 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:30.527+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:30.597 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:30.527+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:30.597 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:30.527+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:30.597 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:30.527+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:30.598 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:30 smithi112 systemd[1]: Stopping Ceph osd.7 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:06:30.598 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:30.516+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:30.598 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:30.516+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:30.598 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:30.516+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:30.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:30.445+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:30.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:30.445+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:30.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:30.445+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:30.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:30.489+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:30.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:30.489+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:30.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:30 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:30.489+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:31.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:30.596+0000 7f12915f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:06:31.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:30.596+0000 7f12915f4700 -1 osd.7 91 *** Got signal Terminated *** 2024-04-25T11:06:31.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:30 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:30.596+0000 7f12915f4700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:31.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:30.784+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:31.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:30.784+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:31.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:30.784+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:31.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:30 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:30.784+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:31.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:31 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:31.486+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:31.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:31 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:31.486+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:31.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:31 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:31.486+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:31.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:31 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:31.475+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:31.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:31 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:31.475+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:31.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:31 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:31.475+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:31.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:31 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:31.474+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:31.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:31 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:31.474+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:31.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:31 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:31.474+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:31.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:31 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:31.521+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:31.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:31 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:31.521+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:31.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:31 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:31.521+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:31.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:31 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:31.521+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:32.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:31 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:31.803+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:32.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:31 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:31.803+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:32.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:31 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:31.803+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:32.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:31 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:31.803+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:32.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:32 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:32.490+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:32.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:32 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:32.490+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:32.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:32 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:32.490+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:32.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:32 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:32.465+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:32.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:32 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:32.465+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:32.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:32 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:32.465+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:32.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:32.475+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:32.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:32.475+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:32.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:32.475+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:32.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:32.554+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:32.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:32.554+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:32.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:32.554+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:32.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:32.554+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:33.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:32 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:32.850+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:33.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:32 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:32.850+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:33.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:32 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:32.851+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:33.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:32 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:32.851+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:33.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:32 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:32.851+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:33.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:33 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:33.516+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:33.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:33 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:33.516+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:33.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:33 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:33.516+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:33.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:33 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:33.442+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:33.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:33 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:33.442+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:33.752 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:33 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:33.442+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:33.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:33 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:33.516+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:33.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:33 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:33.516+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:33.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:33 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:33.516+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:33.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:33 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:33.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:33.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:33 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:33.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:33.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:33 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:33.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:33.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:33 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:33.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:34.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:33 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:33.815+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:34.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:33 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:33.815+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:34.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:33 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:33.815+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:34.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:33 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:33.815+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:34.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:33 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:33.815+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:34.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:34 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:34.543+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:34.752 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:34 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:34.543+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:34.752 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:34 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:34.543+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:34.752 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:34 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:34.403+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:34.752 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:34 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:34.404+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:34.753 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:34 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:34.404+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:34.753 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:34 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:34.404+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:34.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:34 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:34.549+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:34.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:34 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:34.549+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:34.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:34 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:34.549+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:34.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:34 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:34.575+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:34.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:34 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:34.575+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:34.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:34 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:34.575+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:34.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:34 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:34.575+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:35.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:34.780+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:35.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:34.780+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:35.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:34.780+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:35.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:34.780+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:35.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:34 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:34.780+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:35.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:35 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:35.536+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:35.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:35 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:35.536+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:35.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:35 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:35.536+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:35.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:35 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:35.449+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:35.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:35 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:35.449+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:35.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:35 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:35.449+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:35.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:35 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:35.449+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:35.808 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:35.550+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:44.302290+0000 front 2024-04-25T11:05:44.302326+0000 (oldest deadline 2024-04-25T11:06:10.202491+0000) 2024-04-25T11:06:35.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:35.550+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:53.004402+0000 front 2024-04-25T11:05:53.004319+0000 (oldest deadline 2024-04-25T11:06:18.304470+0000) 2024-04-25T11:06:35.809 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7[50760]: 2024-04-25T11:06:35.550+0000 7f128dbaa700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.305118+0000 front 2024-04-25T11:05:58.305211+0000 (oldest deadline 2024-04-25T11:06:27.105620+0000) 2024-04-25T11:06:35.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:35.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:35.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:35.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:35.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:35.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:35.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:35.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:35.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:35 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:35.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:36.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:35.798+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:36.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:35.798+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:36.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:35.798+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:36.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:35.799+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:36.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:35 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:35.799+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:36.342 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:36 smithi112 podman[68555]: 2024-04-25 11:06:36.063405562 +0000 UTC m=+5.591645000 container died 25911ca55c3572d0e996a4477a37d97dbfcbc2f5ca4d8a3e36fae63c128535c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:06:36.717 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:36 smithi112 podman[68555]: 2024-04-25 11:06:36.449731369 +0000 UTC m=+5.977970805 container cleanup 25911ca55c3572d0e996a4477a37d97dbfcbc2f5ca4d8a3e36fae63c128535c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T11:06:36.717 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:36 smithi112 bash[68555]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7 2024-04-25T11:06:36.717 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:36 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:36.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:36.717 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:36 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:36.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:36.717 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:36 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:36.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:36.718 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:36 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:36.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:36.718 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:36 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:36.529+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:36.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:36 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:36.564+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:36.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:36 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:36.564+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:36.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:36 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:36.564+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:36.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:36 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:36.413+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:36.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:36 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:36.413+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:36.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:36 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:36.413+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:36.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:36 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:36.413+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:37.058 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:36 smithi112 podman[68573]: 2024-04-25 11:06:36.717625832 +0000 UTC m=+0.644987631 container remove 25911ca55c3572d0e996a4477a37d97dbfcbc2f5ca4d8a3e36fae63c128535c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:06:37.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:36.776+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:37.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:36.776+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:37.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:36.776+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:37.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:36.776+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:37.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:36 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:36.776+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:37.329 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:37 smithi112 podman[68662]: 2024-04-25 11:06:36.971893774 +0000 UTC m=+0.018051129 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:06:37.330 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:37 smithi112 podman[68662]: 2024-04-25 11:06:37.085174676 +0000 UTC m=+0.131332031 container create 396b02ac6792d483e88ba4f4f82326a4429e4e3f672e280db411029c72d9759c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:06:37.330 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:37 smithi112 podman[68662]: 2024-04-25 11:06:37.329441886 +0000 UTC m=+0.375599247 container init 396b02ac6792d483e88ba4f4f82326a4429e4e3f672e280db411029c72d9759c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T11:06:37.628 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:37 smithi112 podman[68662]: 2024-04-25 11:06:37.332995913 +0000 UTC m=+0.379153269 container start 396b02ac6792d483e88ba4f4f82326a4429e4e3f672e280db411029c72d9759c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T11:06:37.628 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:37 smithi112 podman[68662]: 2024-04-25 11:06:37.429126562 +0000 UTC m=+0.475283930 container attach 396b02ac6792d483e88ba4f4f82326a4429e4e3f672e280db411029c72d9759c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T11:06:37.629 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:37 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:37.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:37.629 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:37 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:37.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:37.629 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:37 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:37.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:37.629 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:37 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:37.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:37.629 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:37 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:37.552+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:37.545+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:37.545+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:37.545+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:37.545+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:09.540031+0000 front 2024-04-25T11:06:09.540118+0000 (oldest deadline 2024-04-25T11:06:37.140369+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:37.396+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:37.397+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:37.397+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:37.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:37 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:37.397+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:37.973 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 11:06:37 smithi112 podman[68662]: 2024-04-25 11:06:37.628478623 +0000 UTC m=+0.674635978 container died 396b02ac6792d483e88ba4f4f82326a4429e4e3f672e280db411029c72d9759c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T11:06:38.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:37.758+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:38.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:37.758+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:38.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:37.758+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:38.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:37.758+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:38.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:37 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:37.758+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:38.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T11:06:38.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:06:38.285 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T11:06:38.285 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T11:06:38.286 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.8 2024-04-25T11:06:38.608 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:38.354+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:38.354+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:38.354+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:38.354+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 systemd[1]: Stopping Ceph osd.8 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:38.502+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:38.502+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:38.502+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:38.502+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:09.540031+0000 front 2024-04-25T11:06:09.540118+0000 (oldest deadline 2024-04-25T11:06:37.140369+0000) 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:38.607+0000 7f2b988f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:06:38.609 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:38.607+0000 7f2b988f6700 -1 osd.8 91 *** Got signal Terminated *** 2024-04-25T11:06:38.610 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:38 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:38.607+0000 7f2b988f6700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:38.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:38 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:38.549+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:38.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:38 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:38.549+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:38.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:38 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:38.549+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:38.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:38 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:38.549+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:38.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:38 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:38.549+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:39.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:38.800+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:39.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:38.800+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:39.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:38.800+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:39.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:38.800+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:39.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:38 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:38.800+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:39.750 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:39.459+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:39.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:39.459+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:39.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:39.459+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:39.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:39.459+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:09.540031+0000 front 2024-04-25T11:06:09.540118+0000 (oldest deadline 2024-04-25T11:06:37.140369+0000) 2024-04-25T11:06:39.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:39.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:39.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:39.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:39.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:39.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:39.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:39 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:39.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:39.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:39 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:39.535+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:39.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:39 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:39.535+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:39.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:39 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:39.535+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:39.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:39 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:39.535+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:39.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:39 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:39.535+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:40.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:39.806+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:40.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:39.806+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:40.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:39.806+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:40.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:39.806+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:40.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:39 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:39.806+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:40.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:40.427+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:40.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:40.427+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:40.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:40.427+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:40.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:40.427+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:09.540031+0000 front 2024-04-25T11:06:09.540118+0000 (oldest deadline 2024-04-25T11:06:37.140369+0000) 2024-04-25T11:06:40.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:40.356+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:40.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:40.356+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:40.752 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:40.356+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:40.752 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:40 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:40.356+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:41.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:40 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:40.558+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:41.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:40 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:40.558+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:41.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:40 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:40.558+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:41.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:40 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:40.558+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:41.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:40 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:40.558+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:41.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:40.771+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:41.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:40.771+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:41.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:40.771+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:41.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:40.771+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:41.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:40 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:40.771+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:41.439+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:41.439+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:41.439+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:41.439+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:09.540031+0000 front 2024-04-25T11:06:09.540118+0000 (oldest deadline 2024-04-25T11:06:37.140369+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:41.312+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:41.312+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:41.312+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:41.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:41 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:41.312+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:41.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:41 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:41.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:41.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:41 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:41.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:41.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:41 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:41.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:41.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:41 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:41.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:41.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:41 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:41.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:41.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:41 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:41.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:41.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:41.759+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:41.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:41.759+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:41.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:41.759+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:41.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:41.759+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:41.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:41.759+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:41.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:41 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:41.759+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:42.651 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:42.408+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:42.651 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:42.408+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:42.651 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:42.408+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:42.651 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:42.408+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:09.540031+0000 front 2024-04-25T11:06:09.540118+0000 (oldest deadline 2024-04-25T11:06:37.140369+0000) 2024-04-25T11:06:42.652 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:42.339+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:42.652 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:42.339+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:42.652 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:42.339+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:42.652 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:42.339+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:42.652 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:42 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:42.339+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:42.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:42.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:42.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:42.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:42.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:42.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:42.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:42.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:42.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:42.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:42.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:42 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:42.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:43.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:42 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:42.748+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:43.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:42 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:42.748+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:43.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:42 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:42.748+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:43.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:42 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:42.748+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:43.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:42 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:42.748+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:43.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:42 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:42.748+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:43.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:43.395+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:35.534949+0000 front 2024-04-25T11:05:35.534883+0000 (oldest deadline 2024-04-25T11:06:00.834803+0000) 2024-04-25T11:06:43.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:43.395+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.838137+0000 front 2024-04-25T11:05:54.838080+0000 (oldest deadline 2024-04-25T11:06:18.338241+0000) 2024-04-25T11:06:43.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:43.395+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:58.338791+0000 front 2024-04-25T11:05:58.338916+0000 (oldest deadline 2024-04-25T11:06:24.238858+0000) 2024-04-25T11:06:43.751 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8[51549]: 2024-04-25T11:06:43.395+0000 7f2b94eac700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:09.540031+0000 front 2024-04-25T11:06:09.540118+0000 (oldest deadline 2024-04-25T11:06:37.140369+0000) 2024-04-25T11:06:43.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:43.315+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:43.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:43.315+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:43.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:43.315+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:43.752 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:43.315+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:43.752 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:43 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:43.315+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:43.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:43.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:43.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:43.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:43.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:43.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:43.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:43.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:43.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:43.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:43.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:43 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:43.544+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:44.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:43 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:43.714+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:44.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:43 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:43.714+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:44.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:43 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:43.714+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:44.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:43 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:43.714+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:44.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:43 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:43.714+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:44.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:43 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:43.714+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:44.250 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:44 smithi169 podman[69947]: 2024-04-25 11:06:44.044400656 +0000 UTC m=+5.564841120 container died e388a2833af751e74229538511dc678ac8c1f0b875646e9aba5d03f60a12b802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T11:06:44.567 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:44 smithi169 podman[69947]: 2024-04-25 11:06:44.383406413 +0000 UTC m=+5.903846863 container cleanup e388a2833af751e74229538511dc678ac8c1f0b875646e9aba5d03f60a12b802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:06:44.568 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:44 smithi169 bash[69947]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8 2024-04-25T11:06:44.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:44.321+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:44.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:44.321+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:44.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:44.321+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:44.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:44.321+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:44.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:44.321+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:44.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:44 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:44.509+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:44.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:44 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:44.509+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:44.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:44 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:44.509+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:44.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:44 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:44.509+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:44.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:44 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:44.509+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:44.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:44 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:44.509+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:45.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:44 smithi169 podman[69978]: 2024-04-25 11:06:44.568112678 +0000 UTC m=+0.524622508 container remove e388a2833af751e74229538511dc678ac8c1f0b875646e9aba5d03f60a12b802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T11:06:45.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:44 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:44.704+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:45.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:44 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:44.704+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:45.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:44 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:44.704+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:45.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:44 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:44.704+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:45.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:44 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:44.704+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:45.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:44 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:44.704+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:45.254 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:45 smithi169 podman[70072]: 2024-04-25 11:06:44.905653339 +0000 UTC m=+0.017988529 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:06:45.254 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:45 smithi169 podman[70072]: 2024-04-25 11:06:45.027299564 +0000 UTC m=+0.139634754 container create 29e7ecd7af4371b7042bf55cf2b78ac172237455ab489ecee2390b5285fe3093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:06:45.254 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:45 smithi169 podman[70072]: 2024-04-25 11:06:45.25461392 +0000 UTC m=+0.366949112 container init 29e7ecd7af4371b7042bf55cf2b78ac172237455ab489ecee2390b5285fe3093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T11:06:45.551 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:45 smithi169 podman[70072]: 2024-04-25 11:06:45.257826378 +0000 UTC m=+0.370161566 container start 29e7ecd7af4371b7042bf55cf2b78ac172237455ab489ecee2390b5285fe3093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:06:45.551 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:45 smithi169 podman[70072]: 2024-04-25 11:06:45.35460093 +0000 UTC m=+0.466936118 container attach 29e7ecd7af4371b7042bf55cf2b78ac172237455ab489ecee2390b5285fe3093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T11:06:45.551 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:45.358+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:45.552 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:45.358+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:45.552 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:45.358+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:45.552 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:45.358+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:45.552 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:45 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:45.358+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:45.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:45.524+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:45.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:45.524+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:45.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:45.524+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:45.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:45.524+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:45.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:45.524+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:45.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:45 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:45.524+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:45.840 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:45 smithi169 podman[70072]: 2024-04-25 11:06:45.552067685 +0000 UTC m=+0.664402877 container died 29e7ecd7af4371b7042bf55cf2b78ac172237455ab489ecee2390b5285fe3093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T11:06:46.092 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 11:06:46 smithi169 podman[70098]: 2024-04-25 11:06:46.082873932 +0000 UTC m=+0.525605912 container remove 29e7ecd7af4371b7042bf55cf2b78ac172237455ab489ecee2390b5285fe3093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T11:06:46.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T11:06:46.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:45 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:45.736+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:46.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:45 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:45.736+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:46.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:45 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:45.736+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:46.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:45 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:45.736+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:46.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:45 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:45.736+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:46.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:45 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:45.736+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:46.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:06:46.134 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T11:06:46.134 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T11:06:46.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.9 2024-04-25T11:06:46.448 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 systemd[1]: Stopping Ceph osd.9 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:06:46.699 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.447+0000 7fd903ff9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:06:46.699 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.447+0000 7fd903ff9700 -1 osd.9 91 *** Got signal Terminated *** 2024-04-25T11:06:46.699 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.447+0000 7fd903ff9700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:46.699 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.698+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:46.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:46.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:46.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:46.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:46.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:46.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:46.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:46.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:46.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:46 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:46.380+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:47.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:46.565+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:47.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:46.565+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:47.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:46.565+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:47.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:46.565+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:47.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:46.565+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:47.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:46 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:46.565+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:47.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.698+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:47.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.698+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:47.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.698+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:47.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.698+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:47.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:46 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:46.698+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:47.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:47 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:47.392+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:47.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:47 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:47.392+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:47.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:47 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:47.392+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:47.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:47 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:47.392+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:47.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:47 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:47.392+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:47.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:47.541+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:47.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:47.541+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:47.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:47.541+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:47.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:47.541+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:47.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:47.541+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:47.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:47 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:47.541+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:48.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:47 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:47.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:48.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:47 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:47.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:48.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:47 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:47.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:48.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:47 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:47.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:48.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:47 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:47.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:48.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:47 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:47.691+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:48.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:48 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:48.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:48.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:48 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:48.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:48.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:48 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:48.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:48.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:48 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:48.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:48.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:48 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:48.393+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:48.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:48.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:48.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:48.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:48.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:48.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:48.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:48.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:48.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:48.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:48.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:48.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:48.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:48 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:48.526+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:49.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:48 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:48.686+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:49.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:48 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:48.686+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:49.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:48 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:48.686+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:49.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:48 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:48.686+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:49.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:48 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:48.686+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:49.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:48 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:48.686+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:49.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:49 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:49.346+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:49.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:49 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:49.346+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:49.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:49 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:49.346+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:49.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:49 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:49.346+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:49.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:49 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:49.346+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:49.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:49.489+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:49.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:49.489+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:49.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:49.489+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:49.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:49.489+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:49.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:49.489+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:49.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:49.489+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:49.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:49 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:49.489+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:50.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:49 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:49.639+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:50.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:49 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:49.639+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:50.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:49 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:49.639+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:50.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:49 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:49.639+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:50.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:49 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:49.639+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:50.111 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:49 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:49.639+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:50.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:50.355+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:50.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:50.355+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:50.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:50.355+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:50.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:50.355+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:50.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:50 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:50.355+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:50.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:50 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:50.502+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:50.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:50 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:50.502+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:50.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:50 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:50.502+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:50.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:50 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:50.502+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:50.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:50 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:50.502+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:50.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:50 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:50.502+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:50.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:50 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:50.502+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:50.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:50 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:50.598+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:39.976869+0000 front 2024-04-25T11:05:39.976959+0000 (oldest deadline 2024-04-25T11:06:01.076855+0000) 2024-04-25T11:06:50.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:50 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:50.598+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.377862+0000 front 2024-04-25T11:05:46.377887+0000 (oldest deadline 2024-04-25T11:06:09.877993+0000) 2024-04-25T11:06:50.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:50 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:50.598+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:54.479542+0000 front 2024-04-25T11:05:54.479711+0000 (oldest deadline 2024-04-25T11:06:19.179512+0000) 2024-04-25T11:06:50.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:50 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:50.598+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:05:59.680168+0000 front 2024-04-25T11:05:59.680251+0000 (oldest deadline 2024-04-25T11:06:25.580228+0000) 2024-04-25T11:06:50.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:50 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:50.598+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.281778+0000 front 2024-04-25T11:06:10.281754+0000 (oldest deadline 2024-04-25T11:06:32.581539+0000) 2024-04-25T11:06:50.861 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:50 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9[64581]: 2024-04-25T11:06:50.598+0000 7fd8fc5a7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.882775+0000 front 2024-04-25T11:06:17.882795+0000 (oldest deadline 2024-04-25T11:06:41.382589+0000) 2024-04-25T11:06:51.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:51 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:51.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:51.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:51 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:51.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:51.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:51 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:51.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:51.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:51 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:51.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:51.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:51 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:51.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:51.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:51 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:51.373+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:51.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:51 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:51.456+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:51.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:51 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:51.456+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:51.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:51 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:51.456+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:51.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:51 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:51.456+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:51.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:51 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:51.456+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:51.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:51 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:51.456+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:51.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:51 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:51.456+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:52.189 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:51 smithi019 podman[94928]: 2024-04-25 11:06:51.933458392 +0000 UTC m=+5.597153748 container died 6c2ec5f6463f9dc16e6c79963d0ea37bce388f68bc8bc0aece9300b4ce250a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T11:06:52.556 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:52 smithi019 podman[94928]: 2024-04-25 11:06:52.272799862 +0000 UTC m=+5.936495221 container cleanup 6c2ec5f6463f9dc16e6c79963d0ea37bce388f68bc8bc0aece9300b4ce250a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T11:06:52.557 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:52 smithi019 bash[94928]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9 2024-04-25T11:06:52.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:52 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:52.349+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:52.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:52 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:52.349+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:52.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:52 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:52.349+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:52.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:52 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:52.349+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:52.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:52 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:52.349+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:52.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:52 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:52.349+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:52.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:52 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:52.422+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:52.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:52 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:52.422+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:52.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:52 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:52.422+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:52.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:52 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:52.422+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:52.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:52 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:52.422+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:52.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:52 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:52.422+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:52.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:52 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:52.422+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:52.860 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:52 smithi019 podman[94947]: 2024-04-25 11:06:52.556776648 +0000 UTC m=+0.620155749 container remove 6c2ec5f6463f9dc16e6c79963d0ea37bce388f68bc8bc0aece9300b4ce250a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:06:53.360 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:53 smithi019 podman[95038]: 2024-04-25 11:06:52.951185893 +0000 UTC m=+0.017765263 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:06:53.360 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:53 smithi019 podman[95038]: 2024-04-25 11:06:53.049240995 +0000 UTC m=+0.115820373 container create c5a81c61cdfeb0108db341bc7199b05901a7c8599658e3ce1a117611fbf0a471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:06:53.719 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:53 smithi019 podman[95038]: 2024-04-25 11:06:53.41717169 +0000 UTC m=+0.483751028 container init c5a81c61cdfeb0108db341bc7199b05901a7c8599658e3ce1a117611fbf0a471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T11:06:53.720 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:53 smithi019 podman[95038]: 2024-04-25 11:06:53.420606961 +0000 UTC m=+0.487186298 container start c5a81c61cdfeb0108db341bc7199b05901a7c8599658e3ce1a117611fbf0a471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T11:06:53.720 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:53 smithi019 podman[95038]: 2024-04-25 11:06:53.509926343 +0000 UTC m=+0.576505685 container attach c5a81c61cdfeb0108db341bc7199b05901a7c8599658e3ce1a117611fbf0a471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T11:06:53.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:53 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:53.318+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:53.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:53 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:53.318+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:53.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:53 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:53.318+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:53.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:53 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:53.318+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:53.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:53 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:53.318+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:53.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:53 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:53.318+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:53.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:53 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:53.434+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:53.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:53 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:53.434+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:53.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:53 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:53.434+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:53.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:53 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:53.434+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:53.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:53 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:53.434+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:53.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:53 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:53.434+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:53.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:53 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:53.434+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:54.062 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 11:06:53 smithi019 podman[95038]: 2024-04-25 11:06:53.720385398 +0000 UTC m=+0.786964749 container died c5a81c61cdfeb0108db341bc7199b05901a7c8599658e3ce1a117611fbf0a471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:06:54.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T11:06:54.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:06:54.398 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T11:06:54.398 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T11:06:54.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.10 2024-04-25T11:06:54.710 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 systemd[1]: Stopping Ceph osd.10 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:06:54.710 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.468+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.468+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.468+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.468+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.468+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.468+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.468+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.709+0000 7f0ac76ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.709+0000 7f0ac76ba700 -1 osd.10 91 *** Got signal Terminated *** 2024-04-25T11:06:54.711 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:54 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:54.709+0000 7f0ac76ba700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:06:54.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:54.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:54.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:54.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:54.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:54.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:54.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:54.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:54.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:54.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:54.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:54.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:55.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:55 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:55.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:55.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:55 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:55.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:55.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:55 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:55.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:55.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:55 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:55.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:55.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:55 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:55.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:55.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:55 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:55.286+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:55.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:55 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:55.492+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:55.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:55 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:55.492+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:55.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:55 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:55.492+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:55.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:55 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:55.492+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:55.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:55 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:55.492+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:55.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:55 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:55.492+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:55.810 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:55 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:55.492+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:56.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:56 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:56.254+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:56.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:56 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:56.254+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:56.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:56 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:56.254+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:56.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:56 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:56.254+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:56.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:56 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:56.254+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:56.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:56 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:56.255+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:56.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:56 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:56.538+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:56.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:56 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:56.538+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:56.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:56 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:56.538+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:56.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:56 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:56.538+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:56.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:56 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:56.538+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:56.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:56 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:56.538+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:56.809 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:56 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:56.538+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:57.500 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:57.223+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:57.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:57.223+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:57.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:57.223+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:57.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:57.223+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:57.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:57.223+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:57.501 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:57.223+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:58.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:57 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:57.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:58.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:57 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:57.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:58.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:57 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:57.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:58.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:57 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:57.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:58.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:57 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:57.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:58.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:57 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:57.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:58.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:57 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:57.561+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:58.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:58 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:58.262+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:58.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:58 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:58.262+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:58.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:58 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:58.262+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:58.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:58 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:58.262+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:58.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:58 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:58.262+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:58.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:58 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:58.262+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:06:59.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:58 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:58.601+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:06:59.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:58 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:58.601+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:06:59.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:58 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:58.601+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:06:59.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:58 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:58.601+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:06:59.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:58 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:58.601+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:06:59.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:58 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:58.601+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:06:59.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:58 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:58.601+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:06:59.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:59 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:59.304+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:06:59.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:59 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:59.304+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:06:59.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:59 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:59.304+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:06:59.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:59 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:59.304+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:06:59.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:59 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:59.304+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:06:59.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:06:59 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:06:59.304+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:00.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:59.615+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:36.359277+0000 front 2024-04-25T11:05:36.359355+0000 (oldest deadline 2024-04-25T11:06:01.059185+0000) 2024-04-25T11:07:00.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:59.615+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.360139+0000 front 2024-04-25T11:05:46.360260+0000 (oldest deadline 2024-04-25T11:06:09.260155+0000) 2024-04-25T11:07:00.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:59.615+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.562043+0000 front 2024-04-25T11:05:55.562018+0000 (oldest deadline 2024-04-25T11:06:17.862071+0000) 2024-04-25T11:07:00.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:59.615+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T11:06:03.163013+0000 front 2024-04-25T11:06:03.163091+0000 (oldest deadline 2024-04-25T11:06:28.463026+0000) 2024-04-25T11:07:00.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:59.615+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:08.963999+0000 front 2024-04-25T11:06:08.964041+0000 (oldest deadline 2024-04-25T11:06:35.364722+0000) 2024-04-25T11:07:00.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:59.615+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:17.065959+0000 front 2024-04-25T11:06:17.066062+0000 (oldest deadline 2024-04-25T11:06:41.165967+0000) 2024-04-25T11:07:00.059 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:06:59 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10[57340]: 2024-04-25T11:06:59.615+0000 7f0abfc68700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:23.367292+0000 front 2024-04-25T11:06:23.367320+0000 (oldest deadline 2024-04-25T11:06:48.067327+0000) 2024-04-25T11:07:00.402 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:00 smithi112 podman[68781]: 2024-04-25 11:07:00.129714787 +0000 UTC m=+5.531035760 container died 58a98328c88f41fe343bb96b3b71df2811dac0b0157cc787c463b98b5db440ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T11:07:00.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:00 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:00.333+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:00.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:00 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:00.333+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:00.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:00 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:00.333+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:00.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:00 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:00.333+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:00.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:00 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:00.333+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:00.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:00 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:00.333+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:00.752 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:00 smithi112 podman[68781]: 2024-04-25 11:07:00.501225696 +0000 UTC m=+5.902546671 container cleanup 58a98328c88f41fe343bb96b3b71df2811dac0b0157cc787c463b98b5db440ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:07:00.752 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:00 smithi112 bash[68781]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10 2024-04-25T11:07:01.058 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:00 smithi112 podman[68799]: 2024-04-25 11:07:00.752235731 +0000 UTC m=+0.622538395 container remove 58a98328c88f41fe343bb96b3b71df2811dac0b0157cc787c463b98b5db440ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T11:07:01.422 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:01 smithi112 podman[68890]: 2024-04-25 11:07:01.011184522 +0000 UTC m=+0.017538538 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:07:01.422 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:01 smithi112 podman[68890]: 2024-04-25 11:07:01.128287539 +0000 UTC m=+0.134641522 container create 2e70071ac1888240a7dc4312108e2cbdd8c586616178d70186207135708937ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T11:07:01.422 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:01 smithi112 podman[68890]: 2024-04-25 11:07:01.338999754 +0000 UTC m=+0.345353748 container init 2e70071ac1888240a7dc4312108e2cbdd8c586616178d70186207135708937ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T11:07:01.422 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:01 smithi112 podman[68890]: 2024-04-25 11:07:01.342161189 +0000 UTC m=+0.348515178 container start 2e70071ac1888240a7dc4312108e2cbdd8c586616178d70186207135708937ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:07:01.422 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:01 smithi112 podman[68890]: 2024-04-25 11:07:01.422120678 +0000 UTC m=+0.428474667 container attach 2e70071ac1888240a7dc4312108e2cbdd8c586616178d70186207135708937ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:07:01.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:01.360+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:01.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:01.360+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:01.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:01.360+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:01.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:01.360+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:01.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:01.360+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:01.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:01.360+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:01.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:01 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:01.360+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T11:06:35.382811+0000 front 2024-04-25T11:06:35.382892+0000 (oldest deadline 2024-04-25T11:07:01.282697+0000) 2024-04-25T11:07:01.808 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 11:07:01 smithi112 podman[68890]: 2024-04-25 11:07:01.626684146 +0000 UTC m=+0.633038133 container died 2e70071ac1888240a7dc4312108e2cbdd8c586616178d70186207135708937ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T11:07:02.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T11:07:02.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:02.249 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T11:07:02.249 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T11:07:02.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.11 2024-04-25T11:07:02.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 systemd[1]: Stopping Ceph osd.11 for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:02.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.399+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:02.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.399+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:02.568 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.399+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:02.569 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.399+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:02.569 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.399+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:02.569 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.399+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:02.569 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.399+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T11:06:35.382811+0000 front 2024-04-25T11:06:35.382892+0000 (oldest deadline 2024-04-25T11:07:01.282697+0000) 2024-04-25T11:07:02.569 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.568+0000 7fada6041700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:02.569 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.568+0000 7fada6041700 -1 osd.11 91 *** Got signal Terminated *** 2024-04-25T11:07:02.569 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:02 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:02.568+0000 7fada6041700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:07:03.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:03.440+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:03.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:03.440+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:03.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:03.440+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:03.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:03.440+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:03.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:03.440+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:03.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:03.440+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:03.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:03 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:03.440+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T11:06:35.382811+0000 front 2024-04-25T11:06:35.382892+0000 (oldest deadline 2024-04-25T11:07:01.282697+0000) 2024-04-25T11:07:04.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:04.428+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:04.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:04.428+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:04.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:04.428+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:04.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:04.428+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:04.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:04.428+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:04.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:04.428+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:04.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:04 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:04.428+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T11:06:35.382811+0000 front 2024-04-25T11:06:35.382892+0000 (oldest deadline 2024-04-25T11:07:01.282697+0000) 2024-04-25T11:07:05.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:05.418+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:05.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:05.418+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:05.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:05.418+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:05.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:05.418+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:05.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:05.418+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:05.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:05.418+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:05.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:05 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:05.418+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T11:06:35.382811+0000 front 2024-04-25T11:06:35.382892+0000 (oldest deadline 2024-04-25T11:07:01.282697+0000) 2024-04-25T11:07:06.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:06.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:06.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:06.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:06.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:06.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:06.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:06.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:06.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:06.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:06.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:06.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:06.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:06 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:06.463+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T11:06:35.382811+0000 front 2024-04-25T11:06:35.382892+0000 (oldest deadline 2024-04-25T11:07:01.282697+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T11:05:40.074349+0000 front 2024-04-25T11:05:40.074331+0000 (oldest deadline 2024-04-25T11:06:05.374287+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T11:05:46.475285+0000 front 2024-04-25T11:05:46.475384+0000 (oldest deadline 2024-04-25T11:06:09.375143+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T11:05:55.776806+0000 front 2024-04-25T11:05:55.777123+0000 (oldest deadline 2024-04-25T11:06:17.477041+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T11:06:10.279584+0000 front 2024-04-25T11:06:10.279666+0000 (oldest deadline 2024-04-25T11:06:34.379497+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T11:06:16.680410+0000 front 2024-04-25T11:06:16.680475+0000 (oldest deadline 2024-04-25T11:06:41.380434+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T11:06:25.982094+0000 front 2024-04-25T11:06:25.982029+0000 (oldest deadline 2024-04-25T11:06:50.681899+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T11:06:35.382811+0000 front 2024-04-25T11:06:35.382892+0000 (oldest deadline 2024-04-25T11:07:01.282697+0000) 2024-04-25T11:07:07.751 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:07 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11[58433]: 2024-04-25T11:07:07.441+0000 7fada25f7700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-04-25T11:06:41.283055+0000 front 2024-04-25T11:06:41.283093+0000 (oldest deadline 2024-04-25T11:07:06.583217+0000) 2024-04-25T11:07:08.377 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:08 smithi169 podman[70177]: 2024-04-25 11:07:08.107743416 +0000 UTC m=+5.663840427 container died 021686a53af953bb55a91c364c3822ffbcd4e3c60ecb355963a10c1b6fd69c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T11:07:08.735 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:08 smithi169 podman[70177]: 2024-04-25 11:07:08.476592519 +0000 UTC m=+6.032689539 container cleanup 021686a53af953bb55a91c364c3822ffbcd4e3c60ecb355963a10c1b6fd69c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T11:07:08.735 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:08 smithi169 bash[70177]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11 2024-04-25T11:07:09.000 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:08 smithi169 podman[70194]: 2024-04-25 11:07:08.735720876 +0000 UTC m=+0.625893717 container remove 021686a53af953bb55a91c364c3822ffbcd4e3c60ecb355963a10c1b6fd69c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T11:07:09.422 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:09 smithi169 podman[70285]: 2024-04-25 11:07:09.095027864 +0000 UTC m=+0.109782168 container create ceefd50ce541dae17db15f3ce46e2aef5754702789c14163e343be2e428e49c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T11:07:09.422 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:09 smithi169 podman[70285]: 2024-04-25 11:07:09.003185191 +0000 UTC m=+0.017939452 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:07:09.422 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:09 smithi169 podman[70285]: 2024-04-25 11:07:09.339939345 +0000 UTC m=+0.354693584 container init ceefd50ce541dae17db15f3ce46e2aef5754702789c14163e343be2e428e49c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T11:07:09.422 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:09 smithi169 podman[70285]: 2024-04-25 11:07:09.343186762 +0000 UTC m=+0.357941000 container start ceefd50ce541dae17db15f3ce46e2aef5754702789c14163e343be2e428e49c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T11:07:09.422 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:09 smithi169 podman[70285]: 2024-04-25 11:07:09.422599999 +0000 UTC m=+0.437354251 container attach ceefd50ce541dae17db15f3ce46e2aef5754702789c14163e343be2e428e49c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T11:07:09.750 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 11:07:09 smithi169 podman[70285]: 2024-04-25 11:07:09.638366187 +0000 UTC m=+0.653120438 container died ceefd50ce541dae17db15f3ce46e2aef5754702789c14163e343be2e428e49c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T11:07:10.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T11:07:10.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:10.360 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T11:07:10.360 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T11:07:10.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.a 2024-04-25T11:07:10.666 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 11:07:10 smithi019 systemd[1]: Stopping Ceph mds.a for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:11.110 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 11:07:10 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-a[71293]: 2024-04-25T11:07:10.664+0000 7f3c8935b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:11.110 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 11:07:10 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-a[71293]: 2024-04-25T11:07:10.664+0000 7f3c8935b700 -1 mds.a *** got signal Terminated *** 2024-04-25T11:07:12.548 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 11:07:12 smithi019 podman[95159]: 2024-04-25 11:07:12.256584381 +0000 UTC m=+1.687464067 container died 85a8cb700f386cb2863fc5bd4c3ee67cb7bf49c3b3b2fad2f05c82dec79da57b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T11:07:12.860 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 11:07:12 smithi019 podman[95159]: 2024-04-25 11:07:12.639588625 +0000 UTC m=+2.070468301 container cleanup 85a8cb700f386cb2863fc5bd4c3ee67cb7bf49c3b3b2fad2f05c82dec79da57b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T11:07:12.860 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 11:07:12 smithi019 bash[95159]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-a 2024-04-25T11:07:13.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T11:07:13.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:13.116 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T11:07:13.116 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T11:07:13.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.d 2024-04-25T11:07:13.443 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 11:07:13 smithi019 systemd[1]: Stopping Ceph mds.d for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:13.860 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 11:07:13 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d[70335]: 2024-04-25T11:07:13.443+0000 7f433f1fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:13.861 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 11:07:13 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d[70335]: 2024-04-25T11:07:13.443+0000 7f433f1fd700 -1 mds.d *** got signal Terminated *** 2024-04-25T11:07:17.110 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 11:07:16 smithi019 podman[95280]: 2024-04-25 11:07:16.832077727 +0000 UTC m=+3.495003337 container died 9fa8ac13a722282daa64aa2898623220d9577955e3d6ff7429dda1d943495037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T11:07:17.584 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 11:07:17 smithi019 podman[95280]: 2024-04-25 11:07:17.21006649 +0000 UTC m=+3.872992106 container cleanup 9fa8ac13a722282daa64aa2898623220d9577955e3d6ff7429dda1d943495037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T11:07:17.584 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 11:07:17 smithi019 bash[95280]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d 2024-04-25T11:07:17.584 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 11:07:17 smithi019 podman[95294]: 2024-04-25 11:07:17.377902544 +0000 UTC m=+0.537114298 container remove 9fa8ac13a722282daa64aa2898623220d9577955e3d6ff7429dda1d943495037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T11:07:17.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T11:07:17.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:17.639 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T11:07:17.639 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T11:07:17.639 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.g 2024-04-25T11:07:17.860 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 11:07:17 smithi019 systemd[1]: Stopping Ceph mds.g for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:18.360 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 11:07:18 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g[70930]: 2024-04-25T11:07:18.005+0000 7ff5ee042700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:18.360 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 11:07:18 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g[70930]: 2024-04-25T11:07:18.005+0000 7ff5ee042700 -1 mds.g *** got signal Terminated *** 2024-04-25T11:07:21.713 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 11:07:21 smithi019 podman[95400]: 2024-04-25 11:07:21.435374415 +0000 UTC m=+3.567871990 container died 1ca4506a6f3aebb7c6e8ef3d924901706207a649c29b9ec6d7148e73dab65759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T11:07:22.105 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 11:07:21 smithi019 podman[95400]: 2024-04-25 11:07:21.813281875 +0000 UTC m=+3.945779450 container cleanup 1ca4506a6f3aebb7c6e8ef3d924901706207a649c29b9ec6d7148e73dab65759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T11:07:22.106 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 11:07:21 smithi019 bash[95400]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-g 2024-04-25T11:07:22.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T11:07:22.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:22.288 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T11:07:22.288 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T11:07:22.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.j 2024-04-25T11:07:22.616 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 11:07:22 smithi019 systemd[1]: Stopping Ceph mds.j for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:23.110 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 11:07:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j[69982]: 2024-04-25T11:07:22.616+0000 7f0c68e73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:23.111 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 11:07:22 smithi019 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j[69982]: 2024-04-25T11:07:22.616+0000 7f0c68e73700 -1 mds.j *** got signal Terminated *** 2024-04-25T11:07:26.360 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 11:07:26 smithi019 podman[95521]: 2024-04-25 11:07:26.076692285 +0000 UTC m=+3.568370725 container died 27ae0acb694b40d180d33f0730f3a83afdbd2ee74cdcecaf42cfcdc3eb500bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T11:07:26.682 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 11:07:26 smithi019 podman[95521]: 2024-04-25 11:07:26.47509713 +0000 UTC m=+3.966775577 container cleanup 27ae0acb694b40d180d33f0730f3a83afdbd2ee74cdcecaf42cfcdc3eb500bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T11:07:26.682 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 11:07:26 smithi019 bash[95521]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-j 2024-04-25T11:07:26.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T11:07:26.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:26.865 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T11:07:26.865 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T11:07:26.865 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.b 2024-04-25T11:07:27.160 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:07:26 smithi112 systemd[1]: Stopping Ceph mds.b for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:27.427 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:07:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b[62588]: 2024-04-25T11:07:27.159+0000 7f09c9633700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:27.427 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:07:27 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b[62588]: 2024-04-25T11:07:27.159+0000 7f09c9633700 -1 mds.b *** got signal Terminated *** 2024-04-25T11:07:31.452 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:07:31 smithi112 podman[68994]: 2024-04-25 11:07:31.172120908 +0000 UTC m=+4.111850263 container died 902bbc03c6c421415831d11331a2893c16adf553c8567c8d360e0f95dcf16675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T11:07:31.808 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:07:31 smithi112 podman[68994]: 2024-04-25 11:07:31.550946532 +0000 UTC m=+4.490675883 container cleanup 902bbc03c6c421415831d11331a2893c16adf553c8567c8d360e0f95dcf16675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T11:07:31.808 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 11:07:31 smithi112 bash[68994]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-b 2024-04-25T11:07:31.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T11:07:32.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:32.019 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T11:07:32.019 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T11:07:32.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.e 2024-04-25T11:07:32.338 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 11:07:32 smithi112 systemd[1]: Stopping Ceph mds.e for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:32.808 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 11:07:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e[61951]: 2024-04-25T11:07:32.338+0000 7fbd04cf2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:32.808 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 11:07:32 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e[61951]: 2024-04-25T11:07:32.338+0000 7fbd04cf2700 -1 mds.e *** got signal Terminated *** 2024-04-25T11:07:35.888 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 11:07:35 smithi112 podman[69125]: 2024-04-25 11:07:35.602546661 +0000 UTC m=+3.364525556 container died 3617a79f9c35483c250f3fbb90ce612a44a216850f6e298bed63c917f89aa5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T11:07:36.238 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 11:07:35 smithi112 podman[69125]: 2024-04-25 11:07:35.988117882 +0000 UTC m=+3.750096778 container cleanup 3617a79f9c35483c250f3fbb90ce612a44a216850f6e298bed63c917f89aa5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:07:36.238 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 11:07:35 smithi112 bash[69125]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-e 2024-04-25T11:07:36.376 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T11:07:36.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:36.408 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T11:07:36.408 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T11:07:36.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.h 2024-04-25T11:07:36.558 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 11:07:36 smithi112 systemd[1]: Stopping Ceph mds.h for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:37.058 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 11:07:36 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h[62275]: 2024-04-25T11:07:36.732+0000 7f1a2f188700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:37.059 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 11:07:36 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h[62275]: 2024-04-25T11:07:36.732+0000 7f1a2f188700 -1 mds.h *** got signal Terminated *** 2024-04-25T11:07:40.216 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 11:07:39 smithi112 podman[69247]: 2024-04-25 11:07:39.895803583 +0000 UTC m=+3.271777268 container died ffa8dbdcee342ade478faccd0b6e12622150ae56c61c7e26fa5446f40e4c4c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T11:07:40.216 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 11:07:40 smithi112 podman[69247]: 2024-04-25 11:07:40.216558177 +0000 UTC m=+3.592531852 container cleanup ffa8dbdcee342ade478faccd0b6e12622150ae56c61c7e26fa5446f40e4c4c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:07:40.558 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 11:07:40 smithi112 bash[69247]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h 2024-04-25T11:07:40.558 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 11:07:40 smithi112 podman[69262]: 2024-04-25 11:07:40.409510044 +0000 UTC m=+0.512332967 container remove ffa8dbdcee342ade478faccd0b6e12622150ae56c61c7e26fa5446f40e4c4c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T11:07:40.634 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T11:07:40.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:40.665 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T11:07:40.665 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T11:07:40.665 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.k 2024-04-25T11:07:40.978 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 11:07:40 smithi112 systemd[1]: Stopping Ceph mds.k for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:41.308 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 11:07:40 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k[61629]: 2024-04-25T11:07:40.977+0000 7fe899026700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:41.308 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 11:07:40 smithi112 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k[61629]: 2024-04-25T11:07:40.977+0000 7fe899026700 -1 mds.k *** got signal Terminated *** 2024-04-25T11:07:43.554 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 11:07:43 smithi112 podman[69369]: 2024-04-25 11:07:43.283298831 +0000 UTC m=+2.405090999 container died 8683e53191b0153b7c93a37c5ed2d2425589bf7be08331c603011fe96142f9f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T11:07:43.808 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 11:07:43 smithi112 podman[69369]: 2024-04-25 11:07:43.653560262 +0000 UTC m=+2.775352430 container cleanup 8683e53191b0153b7c93a37c5ed2d2425589bf7be08331c603011fe96142f9f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T11:07:43.809 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 11:07:43 smithi112 bash[69369]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-k 2024-04-25T11:07:44.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T11:07:44.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:44.065 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T11:07:44.065 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T11:07:44.065 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.c 2024-04-25T11:07:44.350 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 11:07:44 smithi169 systemd[1]: Stopping Ceph mds.c for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:44.614 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 11:07:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c[63602]: 2024-04-25T11:07:44.349+0000 7f3b00a97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:44.614 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 11:07:44 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c[63602]: 2024-04-25T11:07:44.349+0000 7f3b00a97700 -1 mds.c *** got signal Terminated *** 2024-04-25T11:07:48.727 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 11:07:48 smithi169 podman[70392]: 2024-04-25 11:07:48.358461714 +0000 UTC m=+4.096097306 container died 14a5ea9b29375a2d508cbf2528bf18dfc88391b00945ba23dbd61aaa1bbac761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T11:07:49.000 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 11:07:48 smithi169 podman[70392]: 2024-04-25 11:07:48.807807867 +0000 UTC m=+4.545443463 container cleanup 14a5ea9b29375a2d508cbf2528bf18dfc88391b00945ba23dbd61aaa1bbac761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T11:07:49.001 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 11:07:48 smithi169 bash[70392]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-c 2024-04-25T11:07:49.229 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T11:07:49.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:49.265 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T11:07:49.265 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T11:07:49.265 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.f 2024-04-25T11:07:49.608 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 11:07:49 smithi169 systemd[1]: Stopping Ceph mds.f for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:50.000 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 11:07:49 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f[63282]: 2024-04-25T11:07:49.608+0000 7fc969efa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:50.001 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 11:07:49 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f[63282]: 2024-04-25T11:07:49.608+0000 7fc969efa700 -1 mds.f *** got signal Terminated *** 2024-04-25T11:07:53.878 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 11:07:53 smithi169 podman[70525]: 2024-04-25 11:07:53.61651355 +0000 UTC m=+4.135548335 container died 716ab5d21bb99c7589f2d07d0a39ef171a4b6a5dbc21b83170f78c77471bbdc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-04-25T11:07:54.236 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 11:07:53 smithi169 podman[70525]: 2024-04-25 11:07:53.977669903 +0000 UTC m=+4.496704687 container cleanup 716ab5d21bb99c7589f2d07d0a39ef171a4b6a5dbc21b83170f78c77471bbdc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T11:07:54.236 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 11:07:53 smithi169 bash[70525]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-f 2024-04-25T11:07:54.401 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T11:07:54.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:54.434 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T11:07:54.434 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T11:07:54.434 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.i 2024-04-25T11:07:54.748 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:07:54 smithi169 systemd[1]: Stopping Ceph mds.i for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:55.000 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:07:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i[63909]: 2024-04-25T11:07:54.748+0000 7ff7ceda0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:55.001 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:07:54 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i[63909]: 2024-04-25T11:07:54.748+0000 7ff7ceda0700 -1 mds.i *** got signal Terminated *** 2024-04-25T11:07:56.249 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:07:55 smithi169 podman[70644]: 2024-04-25 11:07:55.996107427 +0000 UTC m=+1.344038022 container died 4dcb3f51c65282f8eeea4eae4f60712fc8752f2601c71d6ce7552fd628ccb7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:07:56.500 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:07:56 smithi169 podman[70644]: 2024-04-25 11:07:56.348571183 +0000 UTC m=+1.696501776 container cleanup 4dcb3f51c65282f8eeea4eae4f60712fc8752f2601c71d6ce7552fd628ccb7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:07:56.501 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 11:07:56 smithi169 bash[70644]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-i 2024-04-25T11:07:56.827 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T11:07:56.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:07:56.858 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T11:07:56.858 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T11:07:56.858 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.l 2024-04-25T11:07:57.168 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 11:07:56 smithi169 systemd[1]: Stopping Ceph mds.l for 9e90b6fe-02f0-11ef-bc93-c7b262605968... 2024-04-25T11:07:57.501 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 11:07:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l[62972]: 2024-04-25T11:07:57.167+0000 7f964c648700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:07:57.501 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 11:07:57 smithi169 ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l[62972]: 2024-04-25T11:07:57.167+0000 7f964c648700 -1 mds.l *** got signal Terminated *** 2024-04-25T11:08:00.750 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 11:08:00 smithi169 podman[70764]: 2024-04-25 11:08:00.398031077 +0000 UTC m=+3.322939199 container died 6ffd791b24c684ac580482b3ee04d0376db93c10ed1b0784618a9cc10de1fe07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:08:01.049 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 11:08:00 smithi169 podman[70764]: 2024-04-25 11:08:00.869845087 +0000 UTC m=+3.794753206 container cleanup 6ffd791b24c684ac580482b3ee04d0376db93c10ed1b0784618a9cc10de1fe07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T11:08:01.049 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 11:08:00 smithi169 bash[70764]: ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968-mds-l 2024-04-25T11:08:01.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e90b6fe-02f0-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T11:08:01.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:08:01.304 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T11:08:01.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:08:01.504 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T11:08:03.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:08:04.125 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T11:08:06.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:08:06.692 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T11:08:08.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:08:09.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:08:09.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:08:09.066 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T11:08:09.068 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi019/crash 2024-04-25T11:08:09.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:08:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:08:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:08:09.114 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi112/crash 2024-04-25T11:08:09.114 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:08:09.150 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:08:09.150 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:08:09.152 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi169/crash 2024-04-25T11:08:09.153 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:08:09.192 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:08:09.192 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:08:09.193 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T11:08:09.193 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T11:08:09.229 INFO:tasks.cephadm:Compressing logs... 2024-04-25T11:08:09.229 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:08:09.272 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:08:09.275 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:08:09.296 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:08:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:: No such file or directory 2024-04-25T11:08:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:08:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T11:08:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T11:08:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T11:08:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.3% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T11:08:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:08:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T11:08:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:08:09.319 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:08:09.321 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:08:09.321 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-04-25T11:08:09.321 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T11:08:09.321 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose --/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T11:08:09.321 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:08:09.321 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T11:08:09.321 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr: 94.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr: 83.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr: 94.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T11:08:09.322 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T11:08:09.323 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:08:09.323 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:08:09.323 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:: No such file or directory 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log: -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T11:08:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T11:08:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T11:08:09.327 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:08:09.327 INFO:teuthology.orchestra.run.smithi169.stderr: 94.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T11:08:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:08:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:08:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:08:09.329 INFO:teuthology.orchestra.run.smithi112.stderr: 94.5% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T11:08:09.591 INFO:teuthology.orchestra.run.smithi169.stderr: 91.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T11:08:09.592 INFO:teuthology.orchestra.run.smithi169.stderr: 95.7% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T11:08:09.653 INFO:teuthology.orchestra.run.smithi019.stderr: 90.1% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T11:08:09.746 INFO:teuthology.orchestra.run.smithi112.stderr: 92.0% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T11:08:10.005 INFO:teuthology.orchestra.run.smithi019.stderr: 93.1% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T11:08:10.009 INFO:teuthology.orchestra.run.smithi169.stderr: 93.5% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T11:08:10.034 INFO:teuthology.orchestra.run.smithi019.stderr: 93.6% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T11:08:10.058 INFO:teuthology.orchestra.run.smithi112.stderr: 92.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T11:08:10.065 INFO:teuthology.orchestra.run.smithi112.stderr: 93.3% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T11:08:10.107 INFO:teuthology.orchestra.run.smithi019.stderr: 92.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T11:08:10.121 INFO:teuthology.orchestra.run.smithi169.stderr: 93.3% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T11:08:10.139 INFO:teuthology.orchestra.run.smithi169.stderr: 93.4% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T11:08:10.153 INFO:teuthology.orchestra.run.smithi019.stderr: 92.9% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T11:08:10.166 INFO:teuthology.orchestra.run.smithi112.stderr: 93.4% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T11:08:10.253 INFO:teuthology.orchestra.run.smithi112.stderr: 93.2% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T11:08:10.353 INFO:teuthology.orchestra.run.smithi019.stderr: 90.5% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T11:08:10.355 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:08:10.355 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m1.070s 2024-04-25T11:08:10.356 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m4.414s 2024-04-25T11:08:10.356 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.161s 2024-04-25T11:08:12.199 INFO:teuthology.orchestra.run.smithi169.stderr: 96.4% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T11:08:12.201 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-25T11:08:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m2.912s 2024-04-25T11:08:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m5.672s 2024-04-25T11:08:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.233s 2024-04-25T11:08:12.826 INFO:teuthology.orchestra.run.smithi112.stderr: 93.4% -- replaced with /var/log/ceph/9e90b6fe-02f0-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T11:08:12.828 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T11:08:12.828 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m3.539s 2024-04-25T11:08:12.828 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m7.113s 2024-04-25T11:08:12.828 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.245s 2024-04-25T11:08:12.829 INFO:tasks.cephadm:Archiving logs... 2024-04-25T11:08:12.829 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi019/log 2024-04-25T11:08:12.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:08:13.455 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi112/log 2024-04-25T11:08:13.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:08:14.486 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi169/log 2024-04-25T11:08:14.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:08:15.222 INFO:tasks.cephadm:Removing cluster... 2024-04-25T11:08:15.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 --force 2024-04-25T11:08:15.423 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T11:08:15.688 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 --force 2024-04-25T11:08:15.869 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T11:08:16.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e90b6fe-02f0-11ef-bc93-c7b262605968 --force 2024-04-25T11:08:16.314 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 9e90b6fe-02f0-11ef-bc93-c7b262605968 2024-04-25T11:08:16.575 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T11:08:16.575 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:08:16.594 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:08:16.609 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:08:16.624 INFO:tasks.cephadm:Teardown complete 2024-04-25T11:08:16.624 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:08:16.638 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:08:16.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:08:16.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:08:16.651 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:08:16.763 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:08:16.774 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:08:16.774 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:08:16.777 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:08:16.780 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:08:16.790 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:08:16.791 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:08:16.793 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:08:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:08:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T11:08:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +906us[ +907us] +/- 41ms 2024-04-25T11:08:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -481us[ -480us] +/- 69ms 2024-04-25T11:08:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -1104us[-1102us] +/- 75ms 2024-04-25T11:08:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:08:16.796 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:08:16.796 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T11:08:16.796 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +916us[ +926us] +/- 41ms 2024-04-25T11:08:16.796 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -473us[ -473us] +/- 69ms 2024-04-25T11:08:16.796 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -1096us[-1086us] +/- 75ms 2024-04-25T11:08:16.797 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:08:16.797 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:08:16.798 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T11:08:16.798 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 +904us[ +916us] +/- 42ms 2024-04-25T11:08:16.798 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -486us[ -475us] +/- 69ms 2024-04-25T11:08:16.798 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -1091us[-1091us] +/- 75ms 2024-04-25T11:08:16.798 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:08:16.799 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:08:16.866 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:08:16.867 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:08:16.877 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:08:16.958 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:08:17.022 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:08:17.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:08:17.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:17.117 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:08:17.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:17.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:08:17.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:17.169 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:08:17.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:08:17.192 INFO:teuthology.task.internal:Duration was 1853.833458 seconds 2024-04-25T11:08:17.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:08:17.204 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:08:17.205 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:08:17.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:08:17.210 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:08:17.243 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:08:17.246 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:08:17.246 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:08:17.650 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:08:17.650 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:08:17.650 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:08:17.714 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T11:08:17.714 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:08:17.742 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:08:17.742 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:08:17.764 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:08:17.765 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:08:17.767 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:08:17.784 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:08:17.829 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:08:17.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:08:17.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:08:17.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:08:18.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:08:18.231 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:08:18.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:08:18.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:08:18.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:08:18.303 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:08:18.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:08:18.318 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:08:18.321 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:08:18.338 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-04-25T11:08:18.341 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-25T11:08:18.365 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-25T11:08:18.390 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:08:18.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:18.405 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:08:18.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:18.419 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:08:18.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:18.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:08:18.456 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:08:18.457 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi019 2024-04-25T11:08:18.457 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:08:18.500 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi112 2024-04-25T11:08:18.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:08:18.535 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673096/remote/smithi169 2024-04-25T11:08:18.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:08:18.570 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:08:18.570 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:18.573 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:18.575 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:18.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:08:18.635 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:08:18.635 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:08:18.647 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:08:18.647 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:08:18.650 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:08:18.652 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:08:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:08 /home/ubuntu/cephtest 2024-04-25T11:08:18.664 INFO:teuthology.orchestra.run.smithi112.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:08 /home/ubuntu/cephtest 2024-04-25T11:08:18.672 INFO:teuthology.orchestra.run.smithi169.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:08 /home/ubuntu/cephtest 2024-04-25T11:08:18.674 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:08:18.685 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:08:18.819 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} duration: 1853.8334579467773 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T11:08:18.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:08:18.897 INFO:teuthology.run:pass